Jul 13 21:59:50.704616 ip-10-0-143-97 systemd[1]: Starting Kubernetes Kubelet... Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.081962 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090004 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090027 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090031 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090035 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090041 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090045 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090049 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:59:51.112828 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090052 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090056 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090059 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090063 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090066 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090073 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090076 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090080 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090083 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090085 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090088 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090091 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090094 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090098 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090101 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090106 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090128 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090241 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090245 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:59:51.117641 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090249 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090252 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090256 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090259 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090262 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090265 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090267 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090270 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090273 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090276 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090282 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090286 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090291 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090293 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090296 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090299 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090302 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090305 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090307 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090310 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:59:51.118240 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090312 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090315 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090317 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090320 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090323 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090326 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090329 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090331 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090334 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090344 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090347 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090350 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090352 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090355 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090358 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090361 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090364 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090367 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090370 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090373 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:59:51.118771 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090376 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090378 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090381 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090384 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090386 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090389 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090391 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090394 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090399 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090401 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090404 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090407 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090409 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090412 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090415 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090417 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090420 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090422 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090425 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:59:51.119343 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090428 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090431 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090434 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090439 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090442 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090446 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090449 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090452 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090454 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090457 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090460 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090462 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090465 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.090467 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091223 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091230 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091233 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091236 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091239 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:59:51.119830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091242 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091245 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091247 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091250 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091252 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091255 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091257 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091260 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091263 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091267 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091269 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091272 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091275 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091278 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091281 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091283 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091286 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091288 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091291 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091293 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:59:51.120381 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091296 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091298 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091301 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091303 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091305 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091308 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091310 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091313 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091315 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091318 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091321 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091326 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091329 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091332 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091334 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091337 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091339 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091342 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091344 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091347 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:59:51.120991 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091350 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091352 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091355 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091357 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091360 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091363 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091367 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091369 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091372 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091375 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091377 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091380 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091382 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091385 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091387 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091390 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091392 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091396 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091400 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:59:51.121568 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091403 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091406 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091409 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091411 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091416 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091419 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091421 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091424 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091428 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091432 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091435 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091438 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091440 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091443 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091445 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091449 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091452 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091455 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091458 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:59:51.122068 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091461 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091465 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091467 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091470 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091473 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091476 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091479 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091481 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091485 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091487 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091490 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091493 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091495 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091498 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091500 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091503 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091919 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091928 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091932 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:59:51.122613 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091935 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091938 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091941 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091944 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091947 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091949 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091951 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091954 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091957 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091959 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091962 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091964 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091967 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091970 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091973 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091975 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091978 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091981 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091984 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091987 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:59:51.320466 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091990 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:59:51.155022 ip-10-0-143-97 systemd[1]: Started Kubernetes Kubelet. Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091993 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091995 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.091998 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092000 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092003 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092006 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092008 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092011 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092013 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092017 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092021 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092024 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092026 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092029 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092031 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092034 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092036 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092040 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:59:51.335907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092043 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092046 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092048 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092051 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092053 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092056 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092058 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092061 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092064 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092067 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092071 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092074 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092076 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092079 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092082 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092085 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092088 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092091 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092093 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092096 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:59:51.336798 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092098 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092101 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092103 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092107 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092112 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092115 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092117 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092119 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092122 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092124 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092127 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092129 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092132 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092134 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092137 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092139 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092142 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092144 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092147 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092149 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:59:51.337919 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092152 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092155 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092158 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092160 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092162 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092165 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092167 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092169 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092172 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092174 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092177 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092179 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092182 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092184 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092187 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092190 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092195 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092296 2570 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092305 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092311 2570 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092315 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 21:59:51.339293 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092319 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092323 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092327 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092331 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092334 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092337 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092341 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092344 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092348 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092351 2570 flags.go:64] FLAG: --cgroup-root="" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092354 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092357 2570 flags.go:64] FLAG: --client-ca-file="" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092360 2570 flags.go:64] FLAG: --cloud-provider="external" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092364 2570 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092368 2570 flags.go:64] FLAG: --cluster-domain="" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092371 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092374 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092377 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092381 2570 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092385 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092388 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092390 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092394 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092397 2570 flags.go:64] FLAG: --contention-profiling="false" Jul 13 21:59:51.340478 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092400 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092403 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092409 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092413 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092417 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092420 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092423 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092426 2570 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092429 2570 flags.go:64] FLAG: --enable-server="true" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092432 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092436 2570 flags.go:64] FLAG: --event-burst="100" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092439 2570 flags.go:64] FLAG: --event-qps="50" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092442 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092445 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092448 2570 flags.go:64] FLAG: --eviction-hard="" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092452 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092455 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092458 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092461 2570 flags.go:64] FLAG: --eviction-soft="" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092463 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092466 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092470 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092473 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092475 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092478 2570 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 21:59:51.341667 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092481 2570 flags.go:64] FLAG: --feature-gates="" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092485 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092488 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092491 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092494 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092497 2570 flags.go:64] FLAG: --healthz-port="10248" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092500 2570 flags.go:64] FLAG: --help="false" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092503 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092506 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092510 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092514 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092517 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092521 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092523 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092526 2570 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092529 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092532 2570 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092535 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092538 2570 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092541 2570 flags.go:64] FLAG: --kube-reserved="" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092543 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092546 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092549 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092552 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 21:59:51.343385 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092555 2570 flags.go:64] FLAG: --lock-file="" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092558 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092561 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092564 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092569 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092572 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092575 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092578 2570 flags.go:64] FLAG: --logging-format="text" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092581 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092584 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092587 2570 flags.go:64] FLAG: --manifest-url="" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092589 2570 flags.go:64] FLAG: --manifest-url-header="" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092594 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092597 2570 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092601 2570 flags.go:64] FLAG: --max-pods="110" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092603 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092607 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092618 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092623 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092626 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092629 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092632 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092640 2570 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092643 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092646 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 21:59:51.344681 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092649 2570 flags.go:64] FLAG: --pod-cidr="" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092652 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092657 2570 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092660 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092663 2570 flags.go:64] FLAG: --pods-per-core="0" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092666 2570 flags.go:64] FLAG: --port="10250" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092669 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092672 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04769b1adef7b2f2b" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092676 2570 flags.go:64] FLAG: --qos-reserved="" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092679 2570 flags.go:64] FLAG: --read-only-port="10255" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092681 2570 flags.go:64] FLAG: --register-node="true" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092684 2570 flags.go:64] FLAG: --register-with-taints="" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092688 2570 flags.go:64] FLAG: --registry-burst="10" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092692 2570 flags.go:64] FLAG: --registry-qps="5" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092695 2570 flags.go:64] FLAG: --reserved-cpus="" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092698 2570 flags.go:64] FLAG: --reserved-memory="" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092702 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092704 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092707 2570 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092710 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092713 2570 flags.go:64] FLAG: --runonce="false" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092716 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092719 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092722 2570 flags.go:64] FLAG: --seccomp-default="false" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092725 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 21:59:51.346334 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092730 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092733 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092736 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092739 2570 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092743 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092745 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092748 2570 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092751 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092754 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092757 2570 flags.go:64] FLAG: --system-cgroups="" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092760 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092765 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092768 2570 flags.go:64] FLAG: --tls-cert-file="" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092771 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092775 2570 flags.go:64] FLAG: --tls-min-version="" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092778 2570 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092781 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092784 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092787 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092790 2570 flags.go:64] FLAG: --v="2" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092794 2570 flags.go:64] FLAG: --version="false" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092798 2570 flags.go:64] FLAG: --vmodule="" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092802 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.092805 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092944 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:59:51.350013 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092948 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092951 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092954 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092957 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092960 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092963 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092965 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092971 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092973 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092976 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092979 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092981 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092984 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092987 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092989 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092992 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092994 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.092997 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093000 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093002 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:59:51.351388 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093004 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093007 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093010 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093013 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093015 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093018 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093020 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093023 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093026 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093029 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093031 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093034 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093036 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093039 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093041 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093044 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093046 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093049 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093052 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:59:51.357355 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093058 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093062 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093066 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093069 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093071 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093075 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093079 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093083 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093086 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093089 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093092 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093095 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093098 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093100 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093103 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093105 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093108 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093111 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:59:51.357984 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093113 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093116 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093118 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093121 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093123 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093127 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093130 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093132 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093135 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093138 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093140 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093143 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093145 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093148 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093153 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093155 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093158 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093161 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093164 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:59:51.358527 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093166 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093169 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093171 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093174 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093176 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093179 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093181 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093183 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093186 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093188 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093191 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093193 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093196 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093198 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093201 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093203 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093206 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093208 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093211 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093214 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:59:51.359254 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093217 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.093219 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.093224 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.099646 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.099665 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.099685 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.099695 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099750 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099756 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099760 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099763 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099766 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099769 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099772 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:59:51.359817 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099775 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099777 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099780 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099782 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099785 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099788 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099791 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099794 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099796 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099799 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099802 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099805 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099809 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099811 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099814 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099818 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099821 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099823 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099826 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099829 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:59:51.360325 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099831 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099834 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099837 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099840 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099843 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099845 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099861 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099866 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099869 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099871 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099874 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099877 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099880 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099882 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099885 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099887 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099890 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099893 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099896 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:59:51.360907 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099899 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099902 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099904 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099907 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099910 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099912 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099915 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099917 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099920 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099922 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099924 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099927 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099930 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099932 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099935 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099938 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099941 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099945 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099948 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099950 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:59:51.585357 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099954 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099958 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099962 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099965 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099969 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099973 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099976 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099979 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099983 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099987 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099990 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099992 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099995 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.099998 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100000 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100003 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100006 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100009 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100012 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:59:51.586830 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100014 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100016 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100019 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100021 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100024 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100027 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100029 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100032 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100034 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100037 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100042 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100044 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100047 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100050 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.100056 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 21:59:51.588231 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100178 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100182 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100185 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100188 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100191 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100194 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100196 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100199 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100202 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100206 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100210 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100212 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100215 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100218 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100221 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100224 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100226 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100229 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100232 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100234 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:59:51.589369 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100236 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100239 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100241 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100244 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100246 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100249 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100251 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100254 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100258 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100261 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100265 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100267 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100270 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100272 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100275 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100278 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100281 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100285 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100288 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:59:51.590298 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100290 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100293 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100296 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100299 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100301 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100304 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100307 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100309 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100312 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100315 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100317 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100320 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100322 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100325 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100327 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100330 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100332 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100335 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100337 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100340 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:59:51.591264 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100342 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100346 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100349 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100353 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100355 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100358 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100361 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100363 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100366 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100368 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100371 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100374 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100376 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100379 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100381 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100384 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100386 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100390 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100392 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:59:51.592203 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100395 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100397 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100400 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100403 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100406 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100408 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100410 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100413 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100415 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100418 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100420 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100423 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100425 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100428 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100433 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100437 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100441 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100444 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100447 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100449 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:59:51.593139 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:51.100452 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.100457 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.100568 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.103289 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.104646 2570 server.go:1019] "Starting client certificate rotation" Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.104747 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.104777 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.107107 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.112715 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.123724 2570 log.go:25] "Validated CRI v1 runtime API" Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.126584 2570 log.go:25] "Validated CRI v1 image API" Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.129835 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 21:59:51.594074 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.132078 2570 fs.go:135] Filesystem UUIDs: map[11025115-65ff-40a7-8cd0-87da358adddf:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 efa505a1-bed5-4fc1-a0b9-1af15b1e32ce:/dev/nvme0n1p3] Jul 13 21:59:51.594783 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.132100 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 21:59:51.594783 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.135391 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.137667 2570 manager.go:216] Machine: {Timestamp:2026-07-13 21:59:51.136320916 +0000 UTC m=+0.319165140 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099466 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29db3e977fa2c4b82d11043726ab56 SystemUUID:ec29db3e-977f-a2c4-b82d-11043726ab56 BootID:0921b773-ffdf-45f3-b1c9-d65ad91864d5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:aa:ea:7a:4c:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:aa:ea:7a:4c:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:e4:e5:59:64:05 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.137768 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.137859 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.140505 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.140528 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-97.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.141622 2570 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.141635 2570 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.141643 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.142313 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.143414 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.145967 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.146084 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.146110 2570 kubelet.go:486] "Attempting to sync node with API server" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.146118 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.146128 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.146137 2570 kubelet.go:398] "Adding apiserver pod source" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.146145 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 21:59:51.595245 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.146499 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.146516 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.148196 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.149871 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.149899 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150035 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150046 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150052 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150057 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150065 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150071 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150587 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150593 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150606 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150611 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150623 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.150631 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.151311 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.151318 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.154314 2570 server.go:1268] "Started kubelet" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.155426 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.155405 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.155476 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.159421 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.160023 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-97.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.160055 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.160287 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-97.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.160628 2570 server.go:310] "Adding debug handlers to kubelet server" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.164013 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.164575 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.164585 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.164683 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165120 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165137 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165307 2570 reconstruct.go:97] "Volume reconstruction finished" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165316 2570 reconciler.go:29] "Reconciler: start to sync state" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165343 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165353 2570 factory.go:55] Registering systemd factory Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165388 2570 factory.go:223] Registration of the systemd container factory successfully Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165595 2570 factory.go:153] Registering CRI-O factory Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.164778 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-97.ec2.internal.18c1f8599b27d8d1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-97.ec2.internal,UID:ip-10-0-143-97.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-97.ec2.internal,},FirstTimestamp:2026-07-13 21:59:51.154284753 +0000 UTC m=+0.337128976,LastTimestamp:2026-07-13 21:59:51.154284753 +0000 UTC m=+0.337128976,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-97.ec2.internal,}" Jul 13 21:59:51.596176 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165609 2570 factory.go:223] Registration of the crio container factory successfully Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165667 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165694 2570 factory.go:103] Registering Raw factory Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.165712 2570 manager.go:1196] Started watching for new ooms in manager Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.166075 2570 manager.go:349] Starting recovery of all containers Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.174085 2570 manager.go:354] Recovery completed Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.176212 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-99ql9" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.176436 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.176477 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-97.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.179179 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-99ql9" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.180081 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.181954 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.181980 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.181991 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.182505 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.182518 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.182529 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.182536 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.183153 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.183174 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.183185 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.184061 2570 policy_none.go:49] "None policy: Start" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.184080 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.184097 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.185470 2570 policy_none.go:47] "Start" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.217657 2570 manager.go:337] "Starting Device Plugin manager" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.217801 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.217811 2570 server.go:87] "Starting device plugin registration server" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.218084 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.218094 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.218195 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.218306 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.218317 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.218808 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.218852 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.308879 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.310160 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.310200 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.310227 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.310296 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.314766 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.318379 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.319220 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.319479 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.319491 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.320162 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.600770 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.350057 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.350081 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-97.ec2.internal\": node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.380114 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.411402 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-97.ec2.internal"] Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.411481 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.413093 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.413120 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.413131 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.414418 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.414568 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.414600 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.415327 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.415339 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.415357 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.415367 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.415356 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.415451 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.417105 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.417130 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.417860 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.417888 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.417902 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.449674 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-97.ec2.internal\" not found" node="ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.456501 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-97.ec2.internal\" not found" node="ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.466890 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fc47430451b075a1351236d9b3ac70ce-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal\" (UID: \"fc47430451b075a1351236d9b3ac70ce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.466913 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc47430451b075a1351236d9b3ac70ce-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal\" (UID: \"fc47430451b075a1351236d9b3ac70ce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.466936 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fa7dc64b698a22b7103fb912a9c68e2a-config\") pod \"kube-apiserver-proxy-ip-10-0-143-97.ec2.internal\" (UID: \"fa7dc64b698a22b7103fb912a9c68e2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.480939 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.567466 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fc47430451b075a1351236d9b3ac70ce-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal\" (UID: \"fc47430451b075a1351236d9b3ac70ce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.602959 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.567495 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc47430451b075a1351236d9b3ac70ce-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal\" (UID: \"fc47430451b075a1351236d9b3ac70ce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.611315 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.567514 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fa7dc64b698a22b7103fb912a9c68e2a-config\") pod \"kube-apiserver-proxy-ip-10-0-143-97.ec2.internal\" (UID: \"fa7dc64b698a22b7103fb912a9c68e2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.611315 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.567562 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fa7dc64b698a22b7103fb912a9c68e2a-config\") pod \"kube-apiserver-proxy-ip-10-0-143-97.ec2.internal\" (UID: \"fa7dc64b698a22b7103fb912a9c68e2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.611315 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.567578 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fc47430451b075a1351236d9b3ac70ce-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal\" (UID: \"fc47430451b075a1351236d9b3ac70ce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.611315 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.567586 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc47430451b075a1351236d9b3ac70ce-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal\" (UID: \"fc47430451b075a1351236d9b3ac70ce\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.611315 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.581595 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:51.812556 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.682315 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:51.812556 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.758966 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.812556 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:51.759186 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-97.ec2.internal" Jul 13 21:59:51.812556 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.783375 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:52.038775 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.884405 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:52.038775 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:51.985311 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:52.290638 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:52.086420 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:52.290638 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.104942 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 21:59:52.290638 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.105128 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 21:59:52.290638 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.182470 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 21:54:51 +0000 UTC" deadline="2028-03-22 00:17:51.905379356 +0000 UTC" Jul 13 21:59:52.290638 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.182503 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14810h17m59.722879801s" Jul 13 21:59:52.290638 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:52.187214 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-97.ec2.internal\" not found" Jul 13 21:59:52.290638 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.230344 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:59:52.290638 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.279154 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:59:52.477870 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.365867 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" Jul 13 21:59:52.477870 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.378968 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 21:59:52.477870 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.379879 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-97.ec2.internal" Jul 13 21:59:52.477870 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.386043 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:59:52.477870 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.388956 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 21:59:52.703310 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:52.512729 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa7dc64b698a22b7103fb912a9c68e2a.slice/crio-de5dcfd6f52c3b7e7dec84b67a0c207a41d50e55fdd3d8d233489afb775c7059 WatchSource:0}: Error finding container de5dcfd6f52c3b7e7dec84b67a0c207a41d50e55fdd3d8d233489afb775c7059: Status 404 returned error can't find the container with id de5dcfd6f52c3b7e7dec84b67a0c207a41d50e55fdd3d8d233489afb775c7059 Jul 13 21:59:52.703310 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:52.513067 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc47430451b075a1351236d9b3ac70ce.slice/crio-9bd22f71d0ebea577b96f4cdb1d48fe399f1ec42aecc0339d0d837dce8302530 WatchSource:0}: Error finding container 9bd22f71d0ebea577b96f4cdb1d48fe399f1ec42aecc0339d0d837dce8302530: Status 404 returned error can't find the container with id 9bd22f71d0ebea577b96f4cdb1d48fe399f1ec42aecc0339d0d837dce8302530 Jul 13 21:59:52.703310 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:52.519455 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.153910 2570 apiserver.go:52] "Watching apiserver" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.159492 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.161703 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-rnlqj","kube-system/global-pull-secret-syncer-4w92z","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b","openshift-image-registry/node-ca-nvk85","openshift-multus/multus-cdxjh","openshift-multus/network-metrics-daemon-l42gj","openshift-network-operator/iptables-alerter-z6dw9","openshift-ovn-kubernetes/ovnkube-node-9grcb","kube-system/konnectivity-agent-nfsn4","kube-system/kube-apiserver-proxy-ip-10-0-143-97.ec2.internal","openshift-cluster-node-tuning-operator/tuned-lvnfs","openshift-dns/node-resolver-c9t96","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal","openshift-multus/multus-additional-cni-plugins-9p4sz"] Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.165469 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z6dw9" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.165547 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.167427 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.167931 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.167986 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.168358 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.168817 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hx8pb\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.169489 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.169602 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-b976d\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.169615 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.169676 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.171709 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nvk85" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.172235 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.172258 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.172247 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.172315 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w85hd\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.174753 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.174889 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.175195 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.175249 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pxknk\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.175271 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.175226 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.178239 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.200316 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mznf4" Jul 13 21:59:53.252600 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.210606 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mznf4" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.254668 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-kubernetes\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.254699 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrlkh\" (UniqueName: \"kubernetes.io/projected/f8dfa967-7e9c-4afd-b834-48dada60d5a8-kube-api-access-jrlkh\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.254724 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-run\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.254747 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-sys\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.254799 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-lib-modules\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.254867 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-var-lib-kubelet\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.254901 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-tuned\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.254926 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/39a30f2f-29f6-4b62-8e07-3c8cda3d16b2-iptables-alerter-script\") pod \"iptables-alerter-z6dw9\" (UID: \"39a30f2f-29f6-4b62-8e07-3c8cda3d16b2\") " pod="openshift-network-operator/iptables-alerter-z6dw9" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.254949 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-sysconfig\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.254980 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-sysctl-d\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.255001 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8dfa967-7e9c-4afd-b834-48dada60d5a8-tmp\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.255026 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39a30f2f-29f6-4b62-8e07-3c8cda3d16b2-host-slash\") pod \"iptables-alerter-z6dw9\" (UID: \"39a30f2f-29f6-4b62-8e07-3c8cda3d16b2\") " pod="openshift-network-operator/iptables-alerter-z6dw9" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.255052 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqzxh\" (UniqueName: \"kubernetes.io/projected/39a30f2f-29f6-4b62-8e07-3c8cda3d16b2-kube-api-access-sqzxh\") pod \"iptables-alerter-z6dw9\" (UID: \"39a30f2f-29f6-4b62-8e07-3c8cda3d16b2\") " pod="openshift-network-operator/iptables-alerter-z6dw9" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.255072 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-modprobe-d\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.255123 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-sysctl-conf\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.255146 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-systemd\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.255744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.255170 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-host\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.352037 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.355799 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-kubernetes\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.355829 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-socket-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.355892 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-run\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.355913 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-sys\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.355934 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-lib-modules\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.355955 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-tuned\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.355947 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-kubernetes\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356029 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-sys\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356077 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-run\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356153 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-lib-modules\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356153 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/39a30f2f-29f6-4b62-8e07-3c8cda3d16b2-iptables-alerter-script\") pod \"iptables-alerter-z6dw9\" (UID: \"39a30f2f-29f6-4b62-8e07-3c8cda3d16b2\") " pod="openshift-network-operator/iptables-alerter-z6dw9" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356187 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-sysctl-d\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356213 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-registration-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356238 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-etc-selinux\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356259 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4fd817a9-f4c3-443d-a8f7-02820f6c426f-serviceca\") pod \"node-ca-nvk85\" (UID: \"4fd817a9-f4c3-443d-a8f7-02820f6c426f\") " pod="openshift-image-registry/node-ca-nvk85" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356281 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrlkh\" (UniqueName: \"kubernetes.io/projected/f8dfa967-7e9c-4afd-b834-48dada60d5a8-kube-api-access-jrlkh\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.380671 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356302 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fd817a9-f4c3-443d-a8f7-02820f6c426f-host\") pod \"node-ca-nvk85\" (UID: \"4fd817a9-f4c3-443d-a8f7-02820f6c426f\") " pod="openshift-image-registry/node-ca-nvk85" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356322 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-sysctl-d\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356370 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39a30f2f-29f6-4b62-8e07-3c8cda3d16b2-host-slash\") pod \"iptables-alerter-z6dw9\" (UID: \"39a30f2f-29f6-4b62-8e07-3c8cda3d16b2\") " pod="openshift-network-operator/iptables-alerter-z6dw9" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356327 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39a30f2f-29f6-4b62-8e07-3c8cda3d16b2-host-slash\") pod \"iptables-alerter-z6dw9\" (UID: \"39a30f2f-29f6-4b62-8e07-3c8cda3d16b2\") " pod="openshift-network-operator/iptables-alerter-z6dw9" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356416 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-sysctl-conf\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356439 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-systemd\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356458 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356471 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qws7\" (UniqueName: \"kubernetes.io/projected/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-kube-api-access-5qws7\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356494 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxj9m\" (UniqueName: \"kubernetes.io/projected/4fd817a9-f4c3-443d-a8f7-02820f6c426f-kube-api-access-rxj9m\") pod \"node-ca-nvk85\" (UID: \"4fd817a9-f4c3-443d-a8f7-02820f6c426f\") " pod="openshift-image-registry/node-ca-nvk85" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356518 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-var-lib-kubelet\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356543 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz9qn\" (UniqueName: \"kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn\") pod \"network-check-target-rnlqj\" (UID: \"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad\") " pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356567 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms9z8\" (UniqueName: \"kubernetes.io/projected/3a422d89-db30-488c-8a59-7282e92cd6ab-kube-api-access-ms9z8\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-sysconfig\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356613 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8dfa967-7e9c-4afd-b834-48dada60d5a8-tmp\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356637 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356659 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-device-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356686 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-sys-fs\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.391969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356711 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqzxh\" (UniqueName: \"kubernetes.io/projected/39a30f2f-29f6-4b62-8e07-3c8cda3d16b2-kube-api-access-sqzxh\") pod \"iptables-alerter-z6dw9\" (UID: \"39a30f2f-29f6-4b62-8e07-3c8cda3d16b2\") " pod="openshift-network-operator/iptables-alerter-z6dw9" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356729 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/39a30f2f-29f6-4b62-8e07-3c8cda3d16b2-iptables-alerter-script\") pod \"iptables-alerter-z6dw9\" (UID: \"39a30f2f-29f6-4b62-8e07-3c8cda3d16b2\") " pod="openshift-network-operator/iptables-alerter-z6dw9" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356739 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-systemd\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356741 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-modprobe-d\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356822 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-modprobe-d\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356834 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-var-lib-kubelet\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356715 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-sysctl-conf\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356837 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-host\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356910 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356921 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-sysconfig\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.356965 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8dfa967-7e9c-4afd-b834-48dada60d5a8-host\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.360207 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8dfa967-7e9c-4afd-b834-48dada60d5a8-tmp\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.360262 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8dfa967-7e9c-4afd-b834-48dada60d5a8-etc-tuned\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.365036 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqzxh\" (UniqueName: \"kubernetes.io/projected/39a30f2f-29f6-4b62-8e07-3c8cda3d16b2-kube-api-access-sqzxh\") pod \"iptables-alerter-z6dw9\" (UID: \"39a30f2f-29f6-4b62-8e07-3c8cda3d16b2\") " pod="openshift-network-operator/iptables-alerter-z6dw9" Jul 13 21:59:53.393512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.365282 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrlkh\" (UniqueName: \"kubernetes.io/projected/f8dfa967-7e9c-4afd-b834-48dada60d5a8-kube-api-access-jrlkh\") pod \"tuned-lvnfs\" (UID: \"f8dfa967-7e9c-4afd-b834-48dada60d5a8\") " pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457358 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-registration-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457401 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-etc-selinux\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457424 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4fd817a9-f4c3-443d-a8f7-02820f6c426f-serviceca\") pod \"node-ca-nvk85\" (UID: \"4fd817a9-f4c3-443d-a8f7-02820f6c426f\") " pod="openshift-image-registry/node-ca-nvk85" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457450 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fd817a9-f4c3-443d-a8f7-02820f6c426f-host\") pod \"node-ca-nvk85\" (UID: \"4fd817a9-f4c3-443d-a8f7-02820f6c426f\") " pod="openshift-image-registry/node-ca-nvk85" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457512 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fd817a9-f4c3-443d-a8f7-02820f6c426f-host\") pod \"node-ca-nvk85\" (UID: \"4fd817a9-f4c3-443d-a8f7-02820f6c426f\") " pod="openshift-image-registry/node-ca-nvk85" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457508 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-registration-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457566 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-etc-selinux\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457572 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qws7\" (UniqueName: \"kubernetes.io/projected/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-kube-api-access-5qws7\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457624 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxj9m\" (UniqueName: \"kubernetes.io/projected/4fd817a9-f4c3-443d-a8f7-02820f6c426f-kube-api-access-rxj9m\") pod \"node-ca-nvk85\" (UID: \"4fd817a9-f4c3-443d-a8f7-02820f6c426f\") " pod="openshift-image-registry/node-ca-nvk85" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457654 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz9qn\" (UniqueName: \"kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn\") pod \"network-check-target-rnlqj\" (UID: \"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad\") " pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457680 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms9z8\" (UniqueName: \"kubernetes.io/projected/3a422d89-db30-488c-8a59-7282e92cd6ab-kube-api-access-ms9z8\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457707 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457729 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-device-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457753 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-sys-fs\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457780 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457805 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-socket-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.503173 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457815 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457845 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-sys-fs\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457917 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-device-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.457970 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.457982 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-socket-dir\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.458058 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs podName:3a422d89-db30-488c-8a59-7282e92cd6ab nodeName:}" failed. No retries permitted until 2026-07-13 21:59:53.958035146 +0000 UTC m=+3.140879360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs") pod "network-metrics-daemon-l42gj" (UID: "3a422d89-db30-488c-8a59-7282e92cd6ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.466773 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.466792 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.466803 2570 projected.go:196] Error preparing data for projected volume kube-api-access-pz9qn for pod openshift-network-diagnostics/network-check-target-rnlqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.466858 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn podName:3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad nodeName:}" failed. No retries permitted until 2026-07-13 21:59:53.966840996 +0000 UTC m=+3.149685208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pz9qn" (UniqueName: "kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn") pod "network-check-target-rnlqj" (UID: "3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.466901 2570 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: object "openshift-multus"/"kube-root-ca.crt" not registered Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.466921 2570 projected.go:291] Couldn't get configMap openshift-multus/openshift-service-ca.crt: object "openshift-multus"/"openshift-service-ca.crt" not registered Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.466936 2570 projected.go:196] Error preparing data for projected volume kube-api-access-ms9z8 for pod openshift-multus/network-metrics-daemon-l42gj: [object "openshift-multus"/"kube-root-ca.crt" not registered, object "openshift-multus"/"openshift-service-ca.crt" not registered] Jul 13 21:59:53.504267 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.467019 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a422d89-db30-488c-8a59-7282e92cd6ab-kube-api-access-ms9z8 podName:3a422d89-db30-488c-8a59-7282e92cd6ab nodeName:}" failed. No retries permitted until 2026-07-13 21:59:53.967005204 +0000 UTC m=+3.149849415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ms9z8" (UniqueName: "kubernetes.io/projected/3a422d89-db30-488c-8a59-7282e92cd6ab-kube-api-access-ms9z8") pod "network-metrics-daemon-l42gj" (UID: "3a422d89-db30-488c-8a59-7282e92cd6ab") : [object "openshift-multus"/"kube-root-ca.crt" not registered, object "openshift-multus"/"openshift-service-ca.crt" not registered] Jul 13 21:59:53.528551 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.528520 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z6dw9" Jul 13 21:59:53.528870 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.528841 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4fd817a9-f4c3-443d-a8f7-02820f6c426f-serviceca\") pod \"node-ca-nvk85\" (UID: \"4fd817a9-f4c3-443d-a8f7-02820f6c426f\") " pod="openshift-image-registry/node-ca-nvk85" Jul 13 21:59:53.529565 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.529487 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:53.529818 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.529784 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 21:59:53.531052 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.531027 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxj9m\" (UniqueName: \"kubernetes.io/projected/4fd817a9-f4c3-443d-a8f7-02820f6c426f-kube-api-access-rxj9m\") pod \"node-ca-nvk85\" (UID: \"4fd817a9-f4c3-443d-a8f7-02820f6c426f\") " pod="openshift-image-registry/node-ca-nvk85" Jul 13 21:59:53.531512 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.531491 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qws7\" (UniqueName: \"kubernetes.io/projected/1c9dc639-01fa-4ffb-8cce-c65ac4dfc493-kube-api-access-5qws7\") pod \"aws-ebs-csi-driver-node-qqf5b\" (UID: \"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.531899 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.531692 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nfsn4" Jul 13 21:59:53.533847 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.533810 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 21:59:53.533847 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.533825 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v44dd\"" Jul 13 21:59:53.534740 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.533930 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 21:59:53.534740 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.534243 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.536276 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.536256 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 21:59:53.536276 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.536270 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 21:59:53.536420 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.536304 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 21:59:53.536420 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.536257 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jl6dz\"" Jul 13 21:59:53.536618 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.536606 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 21:59:53.536736 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.536709 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 21:59:53.536967 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.536953 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.539535 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.539304 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 21:59:53.539535 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.539419 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 21:59:53.540104 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.539708 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 21:59:53.540104 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.539738 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 21:59:53.540104 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.539774 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 21:59:53.540104 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.539997 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nmw8g\"" Jul 13 21:59:53.540104 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.540002 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c9t96" Jul 13 21:59:53.541076 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:53.541052 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a30f2f_29f6_4b62_8e07_3c8cda3d16b2.slice/crio-1025e60d9bad65634119e0226cd283612c8945888b36790cd4039a02a9ee85ec WatchSource:0}: Error finding container 1025e60d9bad65634119e0226cd283612c8945888b36790cd4039a02a9ee85ec: Status 404 returned error can't find the container with id 1025e60d9bad65634119e0226cd283612c8945888b36790cd4039a02a9ee85ec Jul 13 21:59:53.541903 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.541870 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 21:59:53.542752 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.542471 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.544797 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.544775 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 21:59:53.544907 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.544790 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 21:59:53.544970 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.544918 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s4gsv\"" Jul 13 21:59:53.545022 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.544997 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 21:59:53.545319 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.545291 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2xbj7\"" Jul 13 21:59:53.545394 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.545287 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-97.ec2.internal" event={"ID":"fa7dc64b698a22b7103fb912a9c68e2a","Type":"ContainerStarted","Data":"de5dcfd6f52c3b7e7dec84b67a0c207a41d50e55fdd3d8d233489afb775c7059"} Jul 13 21:59:53.545446 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.545416 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:53.545511 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.545489 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 21:59:53.550868 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.550840 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" event={"ID":"fc47430451b075a1351236d9b3ac70ce","Type":"ContainerStarted","Data":"9bd22f71d0ebea577b96f4cdb1d48fe399f1ec42aecc0339d0d837dce8302530"} Jul 13 21:59:53.555075 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.555056 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" Jul 13 21:59:53.560612 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.560595 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" Jul 13 21:59:53.564104 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:53.564080 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8dfa967_7e9c_4afd_b834_48dada60d5a8.slice/crio-a589eb4c7e780de4aca1e9d188118453ab7e3e210a056fb557799d732f903bff WatchSource:0}: Error finding container a589eb4c7e780de4aca1e9d188118453ab7e3e210a056fb557799d732f903bff: Status 404 returned error can't find the container with id a589eb4c7e780de4aca1e9d188118453ab7e3e210a056fb557799d732f903bff Jul 13 21:59:53.566052 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.566031 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 21:59:53.569488 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:53.569440 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c9dc639_01fa_4ffb_8cce_c65ac4dfc493.slice/crio-9e0c3d31e8b61416973e103ab3ea8e7c99c6a20f9731936732e5f7adb8e5330c WatchSource:0}: Error finding container 9e0c3d31e8b61416973e103ab3ea8e7c99c6a20f9731936732e5f7adb8e5330c: Status 404 returned error can't find the container with id 9e0c3d31e8b61416973e103ab3ea8e7c99c6a20f9731936732e5f7adb8e5330c Jul 13 21:59:53.579410 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.579391 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nvk85" Jul 13 21:59:53.590226 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:53.590200 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd817a9_f4c3_443d_a8f7_02820f6c426f.slice/crio-b2635a93b00d1f20728e8262b632c483fe9afb016f397723b50c98d5694af098 WatchSource:0}: Error finding container b2635a93b00d1f20728e8262b632c483fe9afb016f397723b50c98d5694af098: Status 404 returned error can't find the container with id b2635a93b00d1f20728e8262b632c483fe9afb016f397723b50c98d5694af098 Jul 13 21:59:53.658210 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658169 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-slash\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.658210 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658215 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-ovn-node-metrics-cert\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.658470 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658237 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqk6p\" (UniqueName: \"kubernetes.io/projected/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-kube-api-access-tqk6p\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.658470 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658253 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-socket-dir-parent\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.658470 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658277 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2tzf\" (UniqueName: \"kubernetes.io/projected/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-kube-api-access-f2tzf\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.658470 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658295 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-ovnkube-config\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.658470 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658310 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-etc-kubernetes\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.658470 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658345 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70c21dbf-84f1-49fa-8c8d-a885d451262a-agent-certs\") pod \"konnectivity-agent-nfsn4\" (UID: \"70c21dbf-84f1-49fa-8c8d-a885d451262a\") " pod="kube-system/konnectivity-agent-nfsn4" Jul 13 21:59:53.658470 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658394 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-cnibin\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.658470 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658437 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-kubelet\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658486 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9418498-5970-4196-a734-4f64df7d4168-cni-binary-copy\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658514 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658543 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-run-ovn\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658574 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-conf-dir\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658602 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-etc-openvswitch\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658629 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-log-socket\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658654 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-cni-netd\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658685 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-run-netns\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658708 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-var-lib-cni-bin\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658736 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-run-netns\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658762 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-cni-binary-copy\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658811 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-var-lib-cni-multus\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.658872 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658866 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-var-lib-kubelet\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.659453 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658907 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-run-multus-certs\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.659453 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.658945 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-dbus\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:53.659453 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659029 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c06f173-22eb-4265-92c2-45838d12f578-tmp-dir\") pod \"node-resolver-c9t96\" (UID: \"8c06f173-22eb-4265-92c2-45838d12f578\") " pod="openshift-dns/node-resolver-c9t96" Jul 13 21:59:53.659453 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659060 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9418498-5970-4196-a734-4f64df7d4168-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.659453 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659101 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-os-release\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.659453 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659152 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c06f173-22eb-4265-92c2-45838d12f578-hosts-file\") pod \"node-resolver-c9t96\" (UID: \"8c06f173-22eb-4265-92c2-45838d12f578\") " pod="openshift-dns/node-resolver-c9t96" Jul 13 21:59:53.659453 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659187 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64vmm\" (UniqueName: \"kubernetes.io/projected/b9418498-5970-4196-a734-4f64df7d4168-kube-api-access-64vmm\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.659453 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659238 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-run-systemd\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.659453 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659284 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-run-openvswitch\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.659453 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659392 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js9zj\" (UniqueName: \"kubernetes.io/projected/8c06f173-22eb-4265-92c2-45838d12f578-kube-api-access-js9zj\") pod \"node-resolver-c9t96\" (UID: \"8c06f173-22eb-4265-92c2-45838d12f578\") " pod="openshift-dns/node-resolver-c9t96" Jul 13 21:59:53.659943 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659484 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-system-cni-dir\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.659943 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659586 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-systemd-units\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.659943 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659623 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-cni-bin\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.659943 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659648 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-env-overrides\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.659943 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659673 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-cnibin\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.659943 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659707 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-hostroot\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.659943 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659733 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:53.659943 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659794 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-kubelet-config\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:53.659943 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659881 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9418498-5970-4196-a734-4f64df7d4168-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.659943 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659912 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-var-lib-openvswitch\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.660378 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.659963 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70c21dbf-84f1-49fa-8c8d-a885d451262a-konnectivity-ca\") pod \"konnectivity-agent-nfsn4\" (UID: \"70c21dbf-84f1-49fa-8c8d-a885d451262a\") " pod="kube-system/konnectivity-agent-nfsn4" Jul 13 21:59:53.660378 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.660001 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-run-ovn-kubernetes\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.660378 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.660027 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-run-k8s-cni-cncf-io\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.660378 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.660063 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.660378 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.660095 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-ovnkube-script-lib\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.660378 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.660120 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-system-cni-dir\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.660378 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.660188 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-os-release\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.660378 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.660212 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-node-log\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.660378 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.660227 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-cni-dir\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.660378 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.660243 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-daemon-config\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.760780 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760681 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-etc-openvswitch\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.760780 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760740 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-log-socket\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.760780 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760767 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-cni-netd\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760793 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-run-netns\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760820 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-var-lib-cni-bin\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760831 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-etc-openvswitch\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760845 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-run-netns\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760891 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-cni-netd\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760903 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-run-netns\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760925 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-cni-binary-copy\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760951 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-var-lib-cni-multus\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760956 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-log-socket\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760960 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-var-lib-cni-bin\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760976 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-var-lib-kubelet\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.760994 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-run-netns\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761023 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-var-lib-kubelet\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761095 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761065 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-var-lib-cni-multus\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761185 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-run-multus-certs\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761223 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-dbus\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761258 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c06f173-22eb-4265-92c2-45838d12f578-tmp-dir\") pod \"node-resolver-c9t96\" (UID: \"8c06f173-22eb-4265-92c2-45838d12f578\") " pod="openshift-dns/node-resolver-c9t96" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761285 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9418498-5970-4196-a734-4f64df7d4168-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761312 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-os-release\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761284 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-run-multus-certs\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761341 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c06f173-22eb-4265-92c2-45838d12f578-hosts-file\") pod \"node-resolver-c9t96\" (UID: \"8c06f173-22eb-4265-92c2-45838d12f578\") " pod="openshift-dns/node-resolver-c9t96" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761369 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64vmm\" (UniqueName: \"kubernetes.io/projected/b9418498-5970-4196-a734-4f64df7d4168-kube-api-access-64vmm\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761395 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-run-systemd\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761419 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-run-openvswitch\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761450 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js9zj\" (UniqueName: \"kubernetes.io/projected/8c06f173-22eb-4265-92c2-45838d12f578-kube-api-access-js9zj\") pod \"node-resolver-c9t96\" (UID: \"8c06f173-22eb-4265-92c2-45838d12f578\") " pod="openshift-dns/node-resolver-c9t96" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761477 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-system-cni-dir\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761502 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-systemd-units\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761528 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-cni-bin\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761547 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-dbus\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761561 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-cni-binary-copy\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761553 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-env-overrides\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.761674 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761566 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c06f173-22eb-4265-92c2-45838d12f578-tmp-dir\") pod \"node-resolver-c9t96\" (UID: \"8c06f173-22eb-4265-92c2-45838d12f578\") " pod="openshift-dns/node-resolver-c9t96" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761599 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-cnibin\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761626 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-hostroot\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761636 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-run-systemd\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761653 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761698 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-kubelet-config\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761730 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9418498-5970-4196-a734-4f64df7d4168-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761758 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-var-lib-openvswitch\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761796 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70c21dbf-84f1-49fa-8c8d-a885d451262a-konnectivity-ca\") pod \"konnectivity-agent-nfsn4\" (UID: \"70c21dbf-84f1-49fa-8c8d-a885d451262a\") " pod="kube-system/konnectivity-agent-nfsn4" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761823 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-run-ovn-kubernetes\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761851 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-run-k8s-cni-cncf-io\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761883 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761892 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-run-openvswitch\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761924 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-ovnkube-script-lib\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761936 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-systemd-units\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761950 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-system-cni-dir\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761959 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-cni-bin\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.762421 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.761637 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c06f173-22eb-4265-92c2-45838d12f578-hosts-file\") pod \"node-resolver-c9t96\" (UID: \"8c06f173-22eb-4265-92c2-45838d12f578\") " pod="openshift-dns/node-resolver-c9t96" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762014 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-system-cni-dir\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762052 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-var-lib-openvswitch\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762071 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-env-overrides\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762226 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-system-cni-dir\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762266 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-os-release\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762293 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-node-log\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762315 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-cni-dir\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762332 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-daemon-config\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762365 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-slash\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762391 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-ovn-node-metrics-cert\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762422 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqk6p\" (UniqueName: \"kubernetes.io/projected/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-kube-api-access-tqk6p\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762450 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-socket-dir-parent\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762476 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2tzf\" (UniqueName: \"kubernetes.io/projected/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-kube-api-access-f2tzf\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762514 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-ovnkube-config\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762541 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-etc-kubernetes\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762579 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70c21dbf-84f1-49fa-8c8d-a885d451262a-agent-certs\") pod \"konnectivity-agent-nfsn4\" (UID: \"70c21dbf-84f1-49fa-8c8d-a885d451262a\") " pod="kube-system/konnectivity-agent-nfsn4" Jul 13 21:59:53.763294 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762605 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-cnibin\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762611 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9418498-5970-4196-a734-4f64df7d4168-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762625 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9418498-5970-4196-a734-4f64df7d4168-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762635 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70c21dbf-84f1-49fa-8c8d-a885d451262a-konnectivity-ca\") pod \"konnectivity-agent-nfsn4\" (UID: \"70c21dbf-84f1-49fa-8c8d-a885d451262a\") " pod="kube-system/konnectivity-agent-nfsn4" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762629 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-kubelet\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762686 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9418498-5970-4196-a734-4f64df7d4168-cni-binary-copy\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762709 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762712 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-os-release\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762717 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-kubelet-config\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762735 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-run-ovn\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762762 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-conf-dir\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762776 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-os-release\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.762790 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762855 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-hostroot\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762855 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-run-ovn-kubernetes\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762877 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-cnibin\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762889 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-node-log\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762905 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.764152 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762945 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-conf-dir\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.762986 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-slash\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763176 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9418498-5970-4196-a734-4f64df7d4168-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.763214 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret podName:a11f9f29-be8e-4926-af3d-f2d4ed6b00d6 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:54.263192724 +0000 UTC m=+3.446036936 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret") pod "global-pull-secret-syncer-4w92z" (UID: "a11f9f29-be8e-4926-af3d-f2d4ed6b00d6") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763246 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-socket-dir-parent\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763287 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-run-ovn\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763389 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9418498-5970-4196-a734-4f64df7d4168-cni-binary-copy\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763426 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-ovnkube-script-lib\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763473 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-host-run-k8s-cni-cncf-io\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763532 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-etc-kubernetes\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763799 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-cni-dir\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763849 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-host-kubelet\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763855 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-multus-daemon-config\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763910 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-ovnkube-config\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.765109 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.763912 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-cnibin\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.766465 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.766408 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-ovn-node-metrics-cert\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.766748 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.766720 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70c21dbf-84f1-49fa-8c8d-a885d451262a-agent-certs\") pod \"konnectivity-agent-nfsn4\" (UID: \"70c21dbf-84f1-49fa-8c8d-a885d451262a\") " pod="kube-system/konnectivity-agent-nfsn4" Jul 13 21:59:53.771162 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.771114 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js9zj\" (UniqueName: \"kubernetes.io/projected/8c06f173-22eb-4265-92c2-45838d12f578-kube-api-access-js9zj\") pod \"node-resolver-c9t96\" (UID: \"8c06f173-22eb-4265-92c2-45838d12f578\") " pod="openshift-dns/node-resolver-c9t96" Jul 13 21:59:53.771744 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.771719 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64vmm\" (UniqueName: \"kubernetes.io/projected/b9418498-5970-4196-a734-4f64df7d4168-kube-api-access-64vmm\") pod \"multus-additional-cni-plugins-9p4sz\" (UID: \"b9418498-5970-4196-a734-4f64df7d4168\") " pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.772311 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.772270 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqk6p\" (UniqueName: \"kubernetes.io/projected/099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8-kube-api-access-tqk6p\") pod \"ovnkube-node-9grcb\" (UID: \"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8\") " pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.772716 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.772688 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2tzf\" (UniqueName: \"kubernetes.io/projected/35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12-kube-api-access-f2tzf\") pod \"multus-cdxjh\" (UID: \"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12\") " pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.886598 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.886563 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nfsn4" Jul 13 21:59:53.893860 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.893830 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9p4sz" Jul 13 21:59:53.899896 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.899867 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 21:59:53.900586 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:53.900556 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c21dbf_84f1_49fa_8c8d_a885d451262a.slice/crio-58601963dfa8c9d17f828de586ac09a56c2d0772ba8d385c824598210c3d1863 WatchSource:0}: Error finding container 58601963dfa8c9d17f828de586ac09a56c2d0772ba8d385c824598210c3d1863: Status 404 returned error can't find the container with id 58601963dfa8c9d17f828de586ac09a56c2d0772ba8d385c824598210c3d1863 Jul 13 21:59:53.906886 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.906833 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c9t96" Jul 13 21:59:53.912675 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.912320 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cdxjh" Jul 13 21:59:53.912880 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:53.912855 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9418498_5970_4196_a734_4f64df7d4168.slice/crio-81babef8b1e3bba9b4c1cd74424329f072c6c13b3c55475eae3332c3e70ff242 WatchSource:0}: Error finding container 81babef8b1e3bba9b4c1cd74424329f072c6c13b3c55475eae3332c3e70ff242: Status 404 returned error can't find the container with id 81babef8b1e3bba9b4c1cd74424329f072c6c13b3c55475eae3332c3e70ff242 Jul 13 21:59:53.921194 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:53.921156 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c06f173_22eb_4265_92c2_45838d12f578.slice/crio-190502d462e4ce1dd9c85d6a31d915e817b8896da3dc3012251a2fd8500d1fc2 WatchSource:0}: Error finding container 190502d462e4ce1dd9c85d6a31d915e817b8896da3dc3012251a2fd8500d1fc2: Status 404 returned error can't find the container with id 190502d462e4ce1dd9c85d6a31d915e817b8896da3dc3012251a2fd8500d1fc2 Jul 13 21:59:53.926367 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:53.926342 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099fa6a3_1b5f_4f2f_b1e3_e72d817dbec8.slice/crio-046589707bd7dc19b33e9cfd566835224e7a87a209cb75bfe78c8990cfd0a1cc WatchSource:0}: Error finding container 046589707bd7dc19b33e9cfd566835224e7a87a209cb75bfe78c8990cfd0a1cc: Status 404 returned error can't find the container with id 046589707bd7dc19b33e9cfd566835224e7a87a209cb75bfe78c8990cfd0a1cc Jul 13 21:59:53.931232 ip-10-0-143-97 kubenswrapper[2570]: W0713 21:59:53.931203 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35af91a5_5eb6_4f8d_8cdf_4903ccc3cf12.slice/crio-995ea4d373d5e9b0d3dd1682130c9bb29fd8405afee8069e5050e02eb7c37430 WatchSource:0}: Error finding container 995ea4d373d5e9b0d3dd1682130c9bb29fd8405afee8069e5050e02eb7c37430: Status 404 returned error can't find the container with id 995ea4d373d5e9b0d3dd1682130c9bb29fd8405afee8069e5050e02eb7c37430 Jul 13 21:59:53.965280 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:53.964558 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:53.965280 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.964755 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:59:53.965280 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:53.964815 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs podName:3a422d89-db30-488c-8a59-7282e92cd6ab nodeName:}" failed. No retries permitted until 2026-07-13 21:59:54.964796742 +0000 UTC m=+4.147640968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs") pod "network-metrics-daemon-l42gj" (UID: "3a422d89-db30-488c-8a59-7282e92cd6ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:59:54.065995 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.065216 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz9qn\" (UniqueName: \"kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn\") pod \"network-check-target-rnlqj\" (UID: \"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad\") " pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 21:59:54.065995 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.065305 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms9z8\" (UniqueName: \"kubernetes.io/projected/3a422d89-db30-488c-8a59-7282e92cd6ab-kube-api-access-ms9z8\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:54.066224 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:54.066033 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:59:54.066224 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:54.066060 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:59:54.066224 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:54.066076 2570 projected.go:196] Error preparing data for projected volume kube-api-access-pz9qn for pod openshift-network-diagnostics/network-check-target-rnlqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:59:54.066224 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:54.066148 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn podName:3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad nodeName:}" failed. No retries permitted until 2026-07-13 21:59:55.066127255 +0000 UTC m=+4.248971472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz9qn" (UniqueName: "kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn") pod "network-check-target-rnlqj" (UID: "3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:59:54.072182 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.072128 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms9z8\" (UniqueName: \"kubernetes.io/projected/3a422d89-db30-488c-8a59-7282e92cd6ab-kube-api-access-ms9z8\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:54.213746 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.213569 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:54:53 +0000 UTC" deadline="2027-12-22 12:44:02.638326495 +0000 UTC" Jul 13 21:59:54.213746 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.213673 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12638h44m8.424662616s" Jul 13 21:59:54.267942 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.267230 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:54.267942 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:54.267454 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:59:54.267942 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:54.267525 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret podName:a11f9f29-be8e-4926-af3d-f2d4ed6b00d6 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:55.267504578 +0000 UTC m=+4.450348790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret") pod "global-pull-secret-syncer-4w92z" (UID: "a11f9f29-be8e-4926-af3d-f2d4ed6b00d6") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:59:54.310984 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.310953 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 21:59:54.311165 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:54.311099 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 21:59:54.321135 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.321056 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z6dw9" event={"ID":"39a30f2f-29f6-4b62-8e07-3c8cda3d16b2","Type":"ContainerStarted","Data":"1025e60d9bad65634119e0226cd283612c8945888b36790cd4039a02a9ee85ec"} Jul 13 21:59:54.323231 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.323203 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" event={"ID":"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8","Type":"ContainerStarted","Data":"046589707bd7dc19b33e9cfd566835224e7a87a209cb75bfe78c8990cfd0a1cc"} Jul 13 21:59:54.325072 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.325046 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nfsn4" event={"ID":"70c21dbf-84f1-49fa-8c8d-a885d451262a","Type":"ContainerStarted","Data":"58601963dfa8c9d17f828de586ac09a56c2d0772ba8d385c824598210c3d1863"} Jul 13 21:59:54.328029 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.328006 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nvk85" event={"ID":"4fd817a9-f4c3-443d-a8f7-02820f6c426f","Type":"ContainerStarted","Data":"b2635a93b00d1f20728e8262b632c483fe9afb016f397723b50c98d5694af098"} Jul 13 21:59:54.330486 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.330455 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" event={"ID":"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493","Type":"ContainerStarted","Data":"9e0c3d31e8b61416973e103ab3ea8e7c99c6a20f9731936732e5f7adb8e5330c"} Jul 13 21:59:54.333997 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.333961 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" event={"ID":"f8dfa967-7e9c-4afd-b834-48dada60d5a8","Type":"ContainerStarted","Data":"a589eb4c7e780de4aca1e9d188118453ab7e3e210a056fb557799d732f903bff"} Jul 13 21:59:54.336972 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.336903 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cdxjh" event={"ID":"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12","Type":"ContainerStarted","Data":"995ea4d373d5e9b0d3dd1682130c9bb29fd8405afee8069e5050e02eb7c37430"} Jul 13 21:59:54.340916 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.340828 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c9t96" event={"ID":"8c06f173-22eb-4265-92c2-45838d12f578","Type":"ContainerStarted","Data":"190502d462e4ce1dd9c85d6a31d915e817b8896da3dc3012251a2fd8500d1fc2"} Jul 13 21:59:54.347029 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.347007 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p4sz" event={"ID":"b9418498-5970-4196-a734-4f64df7d4168","Type":"ContainerStarted","Data":"81babef8b1e3bba9b4c1cd74424329f072c6c13b3c55475eae3332c3e70ff242"} Jul 13 21:59:54.973180 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:54.973142 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:54.973710 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:54.973300 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:59:54.973710 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:54.973376 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs podName:3a422d89-db30-488c-8a59-7282e92cd6ab nodeName:}" failed. No retries permitted until 2026-07-13 21:59:56.973354013 +0000 UTC m=+6.156198248 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs") pod "network-metrics-daemon-l42gj" (UID: "3a422d89-db30-488c-8a59-7282e92cd6ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:59:55.074349 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:55.074312 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz9qn\" (UniqueName: \"kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn\") pod \"network-check-target-rnlqj\" (UID: \"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad\") " pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 21:59:55.074530 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:55.074514 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:59:55.074592 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:55.074535 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:59:55.074592 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:55.074548 2570 projected.go:196] Error preparing data for projected volume kube-api-access-pz9qn for pod openshift-network-diagnostics/network-check-target-rnlqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:59:55.074746 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:55.074654 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn podName:3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad nodeName:}" failed. No retries permitted until 2026-07-13 21:59:57.074590157 +0000 UTC m=+6.257434386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz9qn" (UniqueName: "kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn") pod "network-check-target-rnlqj" (UID: "3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:59:55.214316 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:55.214215 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:54:53 +0000 UTC" deadline="2028-02-29 20:34:28.993699916 +0000 UTC" Jul 13 21:59:55.214316 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:55.214262 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14302h34m33.779443761s" Jul 13 21:59:55.275971 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:55.275931 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:55.276170 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:55.276091 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:59:55.276170 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:55.276160 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret podName:a11f9f29-be8e-4926-af3d-f2d4ed6b00d6 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:57.276139592 +0000 UTC m=+6.458983818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret") pod "global-pull-secret-syncer-4w92z" (UID: "a11f9f29-be8e-4926-af3d-f2d4ed6b00d6") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:59:55.314625 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:55.314064 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:55.314625 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:55.314199 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 21:59:55.315237 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:55.315048 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:55.315237 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:55.315172 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 21:59:56.311398 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:56.310808 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 21:59:56.311398 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:56.310990 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 21:59:56.990969 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:56.990934 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:56.991194 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:56.991174 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:59:56.991281 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:56.991245 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs podName:3a422d89-db30-488c-8a59-7282e92cd6ab nodeName:}" failed. No retries permitted until 2026-07-13 22:00:00.991225741 +0000 UTC m=+10.174069968 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs") pod "network-metrics-daemon-l42gj" (UID: "3a422d89-db30-488c-8a59-7282e92cd6ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:59:57.091477 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:57.091436 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz9qn\" (UniqueName: \"kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn\") pod \"network-check-target-rnlqj\" (UID: \"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad\") " pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 21:59:57.091767 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:57.091745 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:59:57.091856 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:57.091783 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:59:57.091856 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:57.091800 2570 projected.go:196] Error preparing data for projected volume kube-api-access-pz9qn for pod openshift-network-diagnostics/network-check-target-rnlqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:59:57.092051 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:57.091874 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn podName:3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad nodeName:}" failed. No retries permitted until 2026-07-13 22:00:01.091852196 +0000 UTC m=+10.274696424 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz9qn" (UniqueName: "kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn") pod "network-check-target-rnlqj" (UID: "3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:59:57.293590 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:57.293489 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:57.293779 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:57.293620 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:59:57.293779 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:57.293685 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret podName:a11f9f29-be8e-4926-af3d-f2d4ed6b00d6 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:01.293667665 +0000 UTC m=+10.476511892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret") pod "global-pull-secret-syncer-4w92z" (UID: "a11f9f29-be8e-4926-af3d-f2d4ed6b00d6") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:59:57.311417 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:57.311373 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:57.311929 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:57.311532 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 21:59:57.312112 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:57.312066 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:57.312268 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:57.312242 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 21:59:58.311156 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:58.310749 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 21:59:58.311156 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:58.310881 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 21:59:59.310902 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:59.310811 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 21:59:59.310902 ip-10-0-143-97 kubenswrapper[2570]: I0713 21:59:59.310821 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 21:59:59.311375 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:59.310969 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 21:59:59.311375 ip-10-0-143-97 kubenswrapper[2570]: E0713 21:59:59.311058 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:00.310868 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:00.310826 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:00.311054 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:00.310979 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:01.023866 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:01.023817 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:01.024035 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:01.024022 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 22:00:01.024099 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:01.024089 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs podName:3a422d89-db30-488c-8a59-7282e92cd6ab nodeName:}" failed. No retries permitted until 2026-07-13 22:00:09.024069224 +0000 UTC m=+18.206913448 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs") pod "network-metrics-daemon-l42gj" (UID: "3a422d89-db30-488c-8a59-7282e92cd6ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 22:00:01.125068 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:01.124970 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz9qn\" (UniqueName: \"kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn\") pod \"network-check-target-rnlqj\" (UID: \"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad\") " pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:01.125248 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:01.125221 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 22:00:01.125318 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:01.125250 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 22:00:01.125318 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:01.125265 2570 projected.go:196] Error preparing data for projected volume kube-api-access-pz9qn for pod openshift-network-diagnostics/network-check-target-rnlqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 22:00:01.125414 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:01.125337 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn podName:3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad nodeName:}" failed. No retries permitted until 2026-07-13 22:00:09.125317159 +0000 UTC m=+18.308161380 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz9qn" (UniqueName: "kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn") pod "network-check-target-rnlqj" (UID: "3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 22:00:01.312912 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:01.312225 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:01.312912 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:01.312385 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:01.312912 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:01.312724 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:01.312912 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:01.312818 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:01.326260 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:01.326231 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:01.326415 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:01.326398 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 22:00:01.326478 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:01.326469 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret podName:a11f9f29-be8e-4926-af3d-f2d4ed6b00d6 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:09.326450732 +0000 UTC m=+18.509294957 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret") pod "global-pull-secret-syncer-4w92z" (UID: "a11f9f29-be8e-4926-af3d-f2d4ed6b00d6") : object "kube-system"/"original-pull-secret" not registered Jul 13 22:00:02.311141 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:02.310988 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:02.311141 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:02.311129 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:03.311123 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:03.310950 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:03.311576 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:03.310967 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:03.311576 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:03.311198 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:03.311576 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:03.311320 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:04.310456 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:04.310420 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:04.310636 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:04.310527 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:05.310651 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:05.310614 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:05.311112 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:05.310748 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:05.311112 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:05.310811 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:05.311112 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:05.310927 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:06.310629 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:06.310597 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:06.310794 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:06.310709 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:07.310533 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:07.310494 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:07.310533 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:07.310493 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:07.310768 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:07.310638 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:07.310835 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:07.310758 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:08.311259 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:08.311167 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:08.311715 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:08.311305 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:09.083340 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:09.083310 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:09.083482 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:09.083457 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 22:00:09.083544 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:09.083532 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs podName:3a422d89-db30-488c-8a59-7282e92cd6ab nodeName:}" failed. No retries permitted until 2026-07-13 22:00:25.083511642 +0000 UTC m=+34.266355868 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs") pod "network-metrics-daemon-l42gj" (UID: "3a422d89-db30-488c-8a59-7282e92cd6ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 22:00:09.184050 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:09.184010 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz9qn\" (UniqueName: \"kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn\") pod \"network-check-target-rnlqj\" (UID: \"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad\") " pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:09.184240 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:09.184198 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 22:00:09.184240 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:09.184218 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 22:00:09.184240 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:09.184228 2570 projected.go:196] Error preparing data for projected volume kube-api-access-pz9qn for pod openshift-network-diagnostics/network-check-target-rnlqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 22:00:09.184397 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:09.184290 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn podName:3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad nodeName:}" failed. No retries permitted until 2026-07-13 22:00:25.184274245 +0000 UTC m=+34.367118454 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz9qn" (UniqueName: "kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn") pod "network-check-target-rnlqj" (UID: "3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 22:00:09.310881 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:09.310778 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:09.311027 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:09.310926 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:09.311261 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:09.310782 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:09.311633 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:09.311357 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:09.385787 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:09.385754 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:09.385992 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:09.385977 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 22:00:09.386058 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:09.386046 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret podName:a11f9f29-be8e-4926-af3d-f2d4ed6b00d6 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:25.386026273 +0000 UTC m=+34.568870493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret") pod "global-pull-secret-syncer-4w92z" (UID: "a11f9f29-be8e-4926-af3d-f2d4ed6b00d6") : object "kube-system"/"original-pull-secret" not registered Jul 13 22:00:10.311334 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:10.311307 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:10.311745 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:10.311414 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:11.311859 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:11.311807 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:11.312341 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:11.311944 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:11.312341 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:11.311996 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:11.312341 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:11.312127 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:12.311138 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:12.311100 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:12.311333 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:12.311226 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:13.311483 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:13.311382 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:13.311947 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:13.311535 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:13.311947 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:13.311924 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:13.312062 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:13.312031 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:13.391302 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:13.391072 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" event={"ID":"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8","Type":"ContainerStarted","Data":"a244bb4c3e37b84773d739cf166ca688c7a6045779468f9cb5794fd0db71130b"} Jul 13 22:00:13.392533 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:13.392503 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" event={"ID":"f8dfa967-7e9c-4afd-b834-48dada60d5a8","Type":"ContainerStarted","Data":"f54a13b2172eca209f55c89ec575b658c1dd90929d57b85fa96910dbc6f506c0"} Jul 13 22:00:13.394164 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:13.394102 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-97.ec2.internal" event={"ID":"fa7dc64b698a22b7103fb912a9c68e2a","Type":"ContainerStarted","Data":"7db734e74da2829ad10717dd4c3a573c439d67a92a105e4418226df0bf721b0b"} Jul 13 22:00:13.396240 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:13.396202 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cdxjh" event={"ID":"35af91a5-5eb6-4f8d-8cdf-4903ccc3cf12","Type":"ContainerStarted","Data":"444e6639c39489a13f455dc43d0c22a05402e4b7e3db955ea9c5d2cecb30597a"} Jul 13 22:00:13.410198 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:13.410057 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lvnfs" podStartSLOduration=2.845682922 podStartE2EDuration="22.410042843s" totalImagesPullingTime="19.564359921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:13.409407269 +0000 UTC m=+22.592251500" watchObservedRunningTime="2026-07-13 22:00:13.410042843 +0000 UTC m=+22.592887136" Jul 13 22:00:13.424015 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:13.423964 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-97.ec2.internal" podStartSLOduration=21.423944718 podStartE2EDuration="21.423944718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:13.42343852 +0000 UTC m=+22.606282751" watchObservedRunningTime="2026-07-13 22:00:13.423944718 +0000 UTC m=+22.606788949" Jul 13 22:00:14.310667 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.310490 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:14.310892 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:14.310735 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:14.399611 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.399576 2570 generic.go:356] "Generic (PLEG): container finished" podID="fc47430451b075a1351236d9b3ac70ce" containerID="d84eff01a055fd87439de156a0b5979ef3ec7f495c92a4b9ad61eb2df9641981" exitCode=0 Jul 13 22:00:14.400495 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.399667 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" event={"ID":"fc47430451b075a1351236d9b3ac70ce","Type":"ContainerDied","Data":"d84eff01a055fd87439de156a0b5979ef3ec7f495c92a4b9ad61eb2df9641981"} Jul 13 22:00:14.401346 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.401319 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c9t96" event={"ID":"8c06f173-22eb-4265-92c2-45838d12f578","Type":"ContainerStarted","Data":"baee80e93762962b88e4bb5af667eeb288167fce43e386c25f713c6974ab0d82"} Jul 13 22:00:14.402947 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.402922 2570 generic.go:356] "Generic (PLEG): container finished" podID="b9418498-5970-4196-a734-4f64df7d4168" containerID="72f1bfebd938e90bf81015f000d5804c2f3d65e61dcbab7565ca084068406fba" exitCode=0 Jul 13 22:00:14.403053 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.403011 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p4sz" event={"ID":"b9418498-5970-4196-a734-4f64df7d4168","Type":"ContainerDied","Data":"72f1bfebd938e90bf81015f000d5804c2f3d65e61dcbab7565ca084068406fba"} Jul 13 22:00:14.406260 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.406216 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:00:14.406594 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.406576 2570 generic.go:356] "Generic (PLEG): container finished" podID="099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8" containerID="f8ca375c6f650b6bcd0153527e1485797fbcd4c11f6c91f0cb98577298a26ffa" exitCode=1 Jul 13 22:00:14.406682 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.406649 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" event={"ID":"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8","Type":"ContainerStarted","Data":"f8a3b5dc1f3d1b8f7365564bbd135fc129fac6f84518d729ddea5d91a07d0897"} Jul 13 22:00:14.406746 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.406729 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" event={"ID":"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8","Type":"ContainerStarted","Data":"e21732b6b20fb69e94f42dd36f98cff3bf558d0f89493e4d8b7a5c6a858f8e6c"} Jul 13 22:00:14.406795 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.406764 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" event={"ID":"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8","Type":"ContainerStarted","Data":"31175170132af372d8601f69c94c17056d40665f4f635f3fd6343a6951b610c5"} Jul 13 22:00:14.406834 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.406794 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" event={"ID":"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8","Type":"ContainerStarted","Data":"e3a0756e5d33a619bfb0feaf41f3a72942b84c1df4a10e1b7ea2d2f20367e71e"} Jul 13 22:00:14.406834 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.406821 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" event={"ID":"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8","Type":"ContainerDied","Data":"f8ca375c6f650b6bcd0153527e1485797fbcd4c11f6c91f0cb98577298a26ffa"} Jul 13 22:00:14.408055 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.408031 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nfsn4" event={"ID":"70c21dbf-84f1-49fa-8c8d-a885d451262a","Type":"ContainerStarted","Data":"c1eddadbc635b21407aec0b5c08a6bae7364e28bed381bbde23767c670aa561f"} Jul 13 22:00:14.409526 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.409499 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nvk85" event={"ID":"4fd817a9-f4c3-443d-a8f7-02820f6c426f","Type":"ContainerStarted","Data":"dbe376f8c386e0f2c91b2ab6376f922eb7db3f5587944df07c803a46592111c1"} Jul 13 22:00:14.411135 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.411113 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" event={"ID":"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493","Type":"ContainerStarted","Data":"d05e8a81b94250e1fa21b3a7db1a140f48c2a14c53687887854c27b03a565068"} Jul 13 22:00:14.414586 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.414551 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cdxjh" podStartSLOduration=3.215068891 podStartE2EDuration="22.414539952s" totalImagesPullingTime="19.199471061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:13.440798995 +0000 UTC m=+22.623643230" watchObservedRunningTime="2026-07-13 22:00:14.414539952 +0000 UTC m=+23.597384182" Jul 13 22:00:14.467422 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.467361 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-c9t96" podStartSLOduration=4.260401339 podStartE2EDuration="23.467344532s" totalImagesPullingTime="19.206943193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:14.466840487 +0000 UTC m=+23.649684728" watchObservedRunningTime="2026-07-13 22:00:14.467344532 +0000 UTC m=+23.650188762" Jul 13 22:00:14.480945 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.480895 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nvk85" podStartSLOduration=8.110685861 podStartE2EDuration="23.480880067s" totalImagesPullingTime="15.370194206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:14.480587806 +0000 UTC m=+23.663432039" watchObservedRunningTime="2026-07-13 22:00:14.480880067 +0000 UTC m=+23.663724297" Jul 13 22:00:14.495220 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.495175 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nfsn4" podStartSLOduration=4.269799701 podStartE2EDuration="23.495159533s" totalImagesPullingTime="19.225359832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:14.494761839 +0000 UTC m=+23.677606083" watchObservedRunningTime="2026-07-13 22:00:14.495159533 +0000 UTC m=+23.678003765" Jul 13 22:00:14.634841 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.634809 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:14.739472 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.739385 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:14.841057 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.841024 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:14.881975 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:14.881943 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 22:00:15.233308 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.233209 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T22:00:14.881959257Z","UUID":"682a140a-8c89-4c55-8e59-5302fd126e1f","Handler":null,"Name":"","Endpoint":""} Jul 13 22:00:15.234780 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.234760 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 22:00:15.234909 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.234800 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 22:00:15.311172 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.311141 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:15.311172 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.311141 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:15.311413 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:15.311289 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:15.311413 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:15.311313 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:15.416083 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.416037 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" event={"ID":"fc47430451b075a1351236d9b3ac70ce","Type":"ContainerStarted","Data":"6fe7640d0ef96baff46fe6b88ab709f266c42f7f795764a3288fe4a335c791a1"} Jul 13 22:00:15.417771 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.417722 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z6dw9" event={"ID":"39a30f2f-29f6-4b62-8e07-3c8cda3d16b2","Type":"ContainerStarted","Data":"24ffa0b06de401ebb650e4edae6dedd00b94081fbe54e237062dad1df145c994"} Jul 13 22:00:15.419630 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.419605 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" event={"ID":"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493","Type":"ContainerStarted","Data":"89e6969e3f0da8057bfb071bb0739868044ad424de67eda421bca131db734221"} Jul 13 22:00:15.442101 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.442047 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-97.ec2.internal" podStartSLOduration=23.442032662 podStartE2EDuration="23.442032662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:15.429728552 +0000 UTC m=+24.612572784" watchObservedRunningTime="2026-07-13 22:00:15.442032662 +0000 UTC m=+24.624876892" Jul 13 22:00:15.442495 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.442463 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z6dw9" podStartSLOduration=9.02281129 podStartE2EDuration="24.442453205s" totalImagesPullingTime="15.419641915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:15.44195976 +0000 UTC m=+24.624803992" watchObservedRunningTime="2026-07-13 22:00:15.442453205 +0000 UTC m=+24.625297435" Jul 13 22:00:15.562564 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.562473 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:15.661485 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:15.661454 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:16.310693 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:16.310668 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:16.310815 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:16.310795 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:16.426185 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:16.426160 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:00:16.427047 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:16.426974 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" event={"ID":"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8","Type":"ContainerStarted","Data":"d225ebd10115e0e943c017fe70ef5bde60c6c7ad2213cc871411b2545dab9c4c"} Jul 13 22:00:16.429118 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:16.429045 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" event={"ID":"1c9dc639-01fa-4ffb-8cce-c65ac4dfc493","Type":"ContainerStarted","Data":"d52a53c9ed0005bfce81381a92dbc5d9c2f94428626a2fb1400933d5525a055c"} Jul 13 22:00:16.446106 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:16.446059 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqf5b" podStartSLOduration=11.951977985 podStartE2EDuration="25.446045858s" totalImagesPullingTime="13.494067873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:16.445339632 +0000 UTC m=+25.628183863" watchObservedRunningTime="2026-07-13 22:00:16.446045858 +0000 UTC m=+25.628890088" Jul 13 22:00:16.717716 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:16.717500 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:16.867067 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:16.867031 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:16.968401 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:16.968303 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:17.034736 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:17.034704 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:17.311579 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:17.311502 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:17.311764 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:17.311502 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:17.311764 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:17.311634 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:17.311764 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:17.311703 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:18.310734 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:18.310693 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:18.311293 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:18.310820 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:18.325303 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:18.325268 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:18.429573 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:18.429471 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:18.526040 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:18.526005 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:18.807653 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:18.807630 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:18.905488 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:18.905264 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:19.059889 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.059867 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nfsn4" Jul 13 22:00:19.060406 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.060386 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nfsn4" Jul 13 22:00:19.311362 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.311272 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:19.312096 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:19.311389 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:19.312096 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.311420 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:19.312096 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:19.311481 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:19.440119 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.440084 2570 generic.go:356] "Generic (PLEG): container finished" podID="b9418498-5970-4196-a734-4f64df7d4168" containerID="d244eb8eec821ad1558189d56c7413aa8295ab43382a0e21e34cf914f3be88d4" exitCode=0 Jul 13 22:00:19.440299 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.440171 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p4sz" event={"ID":"b9418498-5970-4196-a734-4f64df7d4168","Type":"ContainerDied","Data":"d244eb8eec821ad1558189d56c7413aa8295ab43382a0e21e34cf914f3be88d4"} Jul 13 22:00:19.443485 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.443466 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:00:19.443862 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.443818 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" event={"ID":"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8","Type":"ContainerStarted","Data":"c9aa1a6352674bdc9501a2ee179b70138229fada92c10f9f11539065ec717315"} Jul 13 22:00:19.444061 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.444040 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nfsn4" Jul 13 22:00:19.444314 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.444277 2570 scope.go:117] "RemoveContainer" containerID="f8ca375c6f650b6bcd0153527e1485797fbcd4c11f6c91f0cb98577298a26ffa" Jul 13 22:00:19.444507 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.444494 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nfsn4" Jul 13 22:00:19.592352 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.592255 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:19.653058 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.653023 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:19.765803 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:19.765767 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:20.311034 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.310998 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:20.311212 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:20.311107 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:20.449500 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.449477 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:00:20.449958 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.449844 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" event={"ID":"099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8","Type":"ContainerStarted","Data":"a1206dbd72d5c2e75f2cde5cba1e03467ae306e715993658b8afa506234a8616"} Jul 13 22:00:20.450066 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.450047 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 22:00:20.450230 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.450215 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 22:00:20.450312 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.450289 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 22:00:20.466044 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.466017 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 22:00:20.466175 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.466126 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 22:00:20.477576 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.477536 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" podStartSLOduration=10.179539274 podStartE2EDuration="29.477522696s" totalImagesPullingTime="19.297983422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:20.477265649 +0000 UTC m=+29.660109880" watchObservedRunningTime="2026-07-13 22:00:20.477522696 +0000 UTC m=+29.660366926" Jul 13 22:00:20.969735 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.969552 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rnlqj"] Jul 13 22:00:20.969922 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.969862 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:20.970000 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:20.969975 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:20.973240 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.973212 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l42gj"] Jul 13 22:00:20.973365 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.973320 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:20.973410 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:20.973398 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:20.973704 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.973678 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4w92z"] Jul 13 22:00:20.973778 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:20.973767 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:20.973876 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:20.973843 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:21.454179 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:21.454141 2570 generic.go:356] "Generic (PLEG): container finished" podID="b9418498-5970-4196-a734-4f64df7d4168" containerID="7ff8a185a09989efeaa92c3f6772011f0e9b9cb1720d3fd423b62b8a95d8f508" exitCode=0 Jul 13 22:00:21.454586 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:21.454224 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p4sz" event={"ID":"b9418498-5970-4196-a734-4f64df7d4168","Type":"ContainerDied","Data":"7ff8a185a09989efeaa92c3f6772011f0e9b9cb1720d3fd423b62b8a95d8f508"} Jul 13 22:00:22.310371 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:22.310346 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:22.310527 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:22.310442 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:22.310527 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:22.310498 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:22.310647 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:22.310618 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:22.925459 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:22.925429 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:23.026764 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:23.026733 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:23.310518 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:23.310488 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:23.310644 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:23.310617 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:23.461334 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:23.461299 2570 generic.go:356] "Generic (PLEG): container finished" podID="b9418498-5970-4196-a734-4f64df7d4168" containerID="e92f28a39b50ecc95b3aa87dd53c8b45b14483f44a8e45b406ff62b45f7fb9e0" exitCode=0 Jul 13 22:00:23.461497 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:23.461339 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p4sz" event={"ID":"b9418498-5970-4196-a734-4f64df7d4168","Type":"ContainerDied","Data":"e92f28a39b50ecc95b3aa87dd53c8b45b14483f44a8e45b406ff62b45f7fb9e0"} Jul 13 22:00:24.310422 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:24.310386 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:24.310422 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:24.310420 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:24.311029 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:24.310523 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92z" podUID="a11f9f29-be8e-4926-af3d-f2d4ed6b00d6" Jul 13 22:00:24.311029 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:24.310660 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l42gj" podUID="3a422d89-db30-488c-8a59-7282e92cd6ab" Jul 13 22:00:25.099090 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:25.099058 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:25.099315 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:25.099229 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 22:00:25.099315 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:25.099297 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs podName:3a422d89-db30-488c-8a59-7282e92cd6ab nodeName:}" failed. No retries permitted until 2026-07-13 22:00:57.099278351 +0000 UTC m=+66.282122563 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs") pod "network-metrics-daemon-l42gj" (UID: "3a422d89-db30-488c-8a59-7282e92cd6ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 22:00:25.200229 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:25.200161 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz9qn\" (UniqueName: \"kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn\") pod \"network-check-target-rnlqj\" (UID: \"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad\") " pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:25.200524 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:25.200341 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 22:00:25.200524 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:25.200362 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 22:00:25.200524 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:25.200371 2570 projected.go:196] Error preparing data for projected volume kube-api-access-pz9qn for pod openshift-network-diagnostics/network-check-target-rnlqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 22:00:25.200524 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:25.200422 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn podName:3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad nodeName:}" failed. No retries permitted until 2026-07-13 22:00:57.200406627 +0000 UTC m=+66.383250836 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pz9qn" (UniqueName: "kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn") pod "network-check-target-rnlqj" (UID: "3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 22:00:25.311038 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:25.311009 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:25.311430 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:25.311128 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rnlqj" podUID="3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad" Jul 13 22:00:25.401467 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:25.401432 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:25.401670 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:25.401599 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 22:00:25.401670 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:25.401662 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret podName:a11f9f29-be8e-4926-af3d-f2d4ed6b00d6 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:57.401644933 +0000 UTC m=+66.584489147 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret") pod "global-pull-secret-syncer-4w92z" (UID: "a11f9f29-be8e-4926-af3d-f2d4ed6b00d6") : object "kube-system"/"original-pull-secret" not registered Jul 13 22:00:25.963346 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:25.963322 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-97.ec2.internal" event="NodeReady" Jul 13 22:00:25.963535 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:25.963450 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 22:00:25.997701 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:25.997670 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85ccff9ddd-tqlj9"] Jul 13 22:00:26.000225 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.000205 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.002239 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.002213 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 22:00:26.002366 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.002254 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 22:00:26.002641 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.002622 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 22:00:26.003187 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.003016 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s74j5\"" Jul 13 22:00:26.012112 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.012081 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 22:00:26.014239 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.014219 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qsg7t"] Jul 13 22:00:26.016210 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.016193 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6b8db"] Jul 13 22:00:26.016327 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.016312 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.017727 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.017710 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6b8db" Jul 13 22:00:26.018264 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.018243 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 22:00:26.018363 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.018271 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lvbhm\"" Jul 13 22:00:26.018666 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.018622 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 22:00:26.020360 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.019669 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 22:00:26.020360 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.019843 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 22:00:26.020360 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.019984 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 22:00:26.020360 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.020206 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-89pgf\"" Jul 13 22:00:26.020822 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.020802 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85ccff9ddd-tqlj9"] Jul 13 22:00:26.027292 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.027269 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6b8db"] Jul 13 22:00:26.029672 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.029651 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qsg7t"] Jul 13 22:00:26.107752 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.107715 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.107752 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.107762 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-certificates\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.108013 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.107801 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-bound-sa-token\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.108013 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.107826 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl88j\" (UniqueName: \"kubernetes.io/projected/bf916935-4397-4211-a340-bdaa676732eb-kube-api-access-dl88j\") pod \"ingress-canary-6b8db\" (UID: \"bf916935-4397-4211-a340-bdaa676732eb\") " pod="openshift-ingress-canary/ingress-canary-6b8db" Jul 13 22:00:26.108013 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.107879 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-config-volume\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.108013 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.107926 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e70b328-f83e-4dec-bb26-b5caab12b0fe-ca-trust-extracted\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.108013 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.107962 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bkr4\" (UniqueName: \"kubernetes.io/projected/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-kube-api-access-7bkr4\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.108013 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.108005 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.108268 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.108024 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76ddw\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-kube-api-access-76ddw\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.108268 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.108084 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-installation-pull-secrets\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.108268 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.108106 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-tmp-dir\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.108268 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.108152 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-image-registry-private-configuration\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.108268 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.108205 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-trusted-ca\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.108268 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.108222 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf916935-4397-4211-a340-bdaa676732eb-cert\") pod \"ingress-canary-6b8db\" (UID: \"bf916935-4397-4211-a340-bdaa676732eb\") " pod="openshift-ingress-canary/ingress-canary-6b8db" Jul 13 22:00:26.167236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.167206 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:26.209171 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209142 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-trusted-ca\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.209355 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209182 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf916935-4397-4211-a340-bdaa676732eb-cert\") pod \"ingress-canary-6b8db\" (UID: \"bf916935-4397-4211-a340-bdaa676732eb\") " pod="openshift-ingress-canary/ingress-canary-6b8db" Jul 13 22:00:26.209355 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209232 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.209355 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209255 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-certificates\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.209355 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209280 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-bound-sa-token\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.209355 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209304 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl88j\" (UniqueName: \"kubernetes.io/projected/bf916935-4397-4211-a340-bdaa676732eb-kube-api-access-dl88j\") pod \"ingress-canary-6b8db\" (UID: \"bf916935-4397-4211-a340-bdaa676732eb\") " pod="openshift-ingress-canary/ingress-canary-6b8db" Jul 13 22:00:26.209355 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209337 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-config-volume\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.209655 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209358 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e70b328-f83e-4dec-bb26-b5caab12b0fe-ca-trust-extracted\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.209655 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209390 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bkr4\" (UniqueName: \"kubernetes.io/projected/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-kube-api-access-7bkr4\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.209655 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209423 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.209655 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209452 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76ddw\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-kube-api-access-76ddw\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.209655 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:26.209472 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 22:00:26.209655 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209489 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-installation-pull-secrets\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.209655 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209519 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-tmp-dir\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.209655 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:26.209569 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls podName:0ca89863-2ae9-49cb-8909-abbc0a5b6bb1 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:26.709538283 +0000 UTC m=+35.892382493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls") pod "dns-default-qsg7t" (UID: "0ca89863-2ae9-49cb-8909-abbc0a5b6bb1") : secret "dns-default-metrics-tls" not found Jul 13 22:00:26.209655 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209599 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-image-registry-private-configuration\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.210149 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.209890 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-tmp-dir\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.210149 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.210032 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-config-volume\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.210149 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.210129 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-certificates\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.210300 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:26.210226 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 22:00:26.210300 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:26.210239 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85ccff9ddd-tqlj9: secret "image-registry-tls" not found Jul 13 22:00:26.210300 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:26.210296 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls podName:7e70b328-f83e-4dec-bb26-b5caab12b0fe nodeName:}" failed. No retries permitted until 2026-07-13 22:00:26.71027466 +0000 UTC m=+35.893118876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls") pod "image-registry-85ccff9ddd-tqlj9" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe") : secret "image-registry-tls" not found Jul 13 22:00:26.210451 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.210320 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e70b328-f83e-4dec-bb26-b5caab12b0fe-ca-trust-extracted\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.210451 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.210375 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-trusted-ca\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.214763 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.214663 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-image-registry-private-configuration\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.214763 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.214686 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-installation-pull-secrets\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.214763 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.214715 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf916935-4397-4211-a340-bdaa676732eb-cert\") pod \"ingress-canary-6b8db\" (UID: \"bf916935-4397-4211-a340-bdaa676732eb\") " pod="openshift-ingress-canary/ingress-canary-6b8db" Jul 13 22:00:26.221050 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.220992 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl88j\" (UniqueName: \"kubernetes.io/projected/bf916935-4397-4211-a340-bdaa676732eb-kube-api-access-dl88j\") pod \"ingress-canary-6b8db\" (UID: \"bf916935-4397-4211-a340-bdaa676732eb\") " pod="openshift-ingress-canary/ingress-canary-6b8db" Jul 13 22:00:26.221050 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.221006 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-bound-sa-token\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.221216 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.221162 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bkr4\" (UniqueName: \"kubernetes.io/projected/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-kube-api-access-7bkr4\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.222484 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.222435 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76ddw\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-kube-api-access-76ddw\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.266511 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.266475 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:26.310605 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.310571 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:26.310780 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.310572 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:26.312908 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.312870 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 22:00:26.313276 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.312916 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 22:00:26.313276 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.312928 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b6tsw\"" Jul 13 22:00:26.335491 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.335472 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6b8db" Jul 13 22:00:26.713197 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.713158 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:26.713424 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:26.713222 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:26.713424 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:26.713322 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 22:00:26.713424 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:26.713394 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls podName:0ca89863-2ae9-49cb-8909-abbc0a5b6bb1 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:27.713374678 +0000 UTC m=+36.896218902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls") pod "dns-default-qsg7t" (UID: "0ca89863-2ae9-49cb-8909-abbc0a5b6bb1") : secret "dns-default-metrics-tls" not found Jul 13 22:00:26.713424 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:26.713324 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 22:00:26.713637 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:26.713429 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85ccff9ddd-tqlj9: secret "image-registry-tls" not found Jul 13 22:00:26.713637 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:26.713482 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls podName:7e70b328-f83e-4dec-bb26-b5caab12b0fe nodeName:}" failed. No retries permitted until 2026-07-13 22:00:27.713470115 +0000 UTC m=+36.896314324 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls") pod "image-registry-85ccff9ddd-tqlj9" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe") : secret "image-registry-tls" not found Jul 13 22:00:27.311209 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.311179 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:27.313623 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.313432 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 22:00:27.313623 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.313430 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gmvbg\"" Jul 13 22:00:27.314177 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.314158 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 22:00:27.349881 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.349646 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:27.403360 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.403335 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:27.495328 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.495296 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6b8db"] Jul 13 22:00:27.499403 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:00:27.499377 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf916935_4397_4211_a340_bdaa676732eb.slice/crio-c64d31ccd9dfa5ce06e12beafd0e4cac13626dff9127415021ffbd6109ad81dc WatchSource:0}: Error finding container c64d31ccd9dfa5ce06e12beafd0e4cac13626dff9127415021ffbd6109ad81dc: Status 404 returned error can't find the container with id c64d31ccd9dfa5ce06e12beafd0e4cac13626dff9127415021ffbd6109ad81dc Jul 13 22:00:27.720734 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.720699 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:27.720955 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.720790 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:27.720955 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:27.720904 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 22:00:27.720955 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:27.720918 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 22:00:27.720955 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:27.720928 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85ccff9ddd-tqlj9: secret "image-registry-tls" not found Jul 13 22:00:27.721115 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:27.720974 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls podName:0ca89863-2ae9-49cb-8909-abbc0a5b6bb1 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:29.720957614 +0000 UTC m=+38.903801843 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls") pod "dns-default-qsg7t" (UID: "0ca89863-2ae9-49cb-8909-abbc0a5b6bb1") : secret "dns-default-metrics-tls" not found Jul 13 22:00:27.721115 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:27.720988 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls podName:7e70b328-f83e-4dec-bb26-b5caab12b0fe nodeName:}" failed. No retries permitted until 2026-07-13 22:00:29.720981917 +0000 UTC m=+38.903826126 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls") pod "image-registry-85ccff9ddd-tqlj9" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe") : secret "image-registry-tls" not found Jul 13 22:00:27.754879 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.754837 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-9sg29"] Jul 13 22:00:27.756917 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.756899 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:27.759202 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.759186 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 13 22:00:27.759348 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.759332 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 13 22:00:27.759572 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.759557 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rxnb2\"" Jul 13 22:00:27.768098 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.768080 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-9sg29"] Jul 13 22:00:27.821880 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.821776 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:27.821880 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.821831 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bec934ce-60f6-47d0-8e29-c4ac675f1225-nginx-conf\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:27.922434 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.922386 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:27.922578 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.922448 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bec934ce-60f6-47d0-8e29-c4ac675f1225-nginx-conf\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:27.922578 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:27.922536 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 22:00:27.922644 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:27.922600 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert podName:bec934ce-60f6-47d0-8e29-c4ac675f1225 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:28.42258547 +0000 UTC m=+37.605429680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9sg29" (UID: "bec934ce-60f6-47d0-8e29-c4ac675f1225") : secret "networking-console-plugin-cert" not found Jul 13 22:00:27.923035 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.923020 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bec934ce-60f6-47d0-8e29-c4ac675f1225-nginx-conf\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:27.969041 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:27.969004 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:28.030621 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.030580 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:28.381686 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.381656 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:28.426408 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.426371 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:28.426576 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:28.426535 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 22:00:28.426646 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:28.426609 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert podName:bec934ce-60f6-47d0-8e29-c4ac675f1225 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:29.42658915 +0000 UTC m=+38.609433376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9sg29" (UID: "bec934ce-60f6-47d0-8e29-c4ac675f1225") : secret "networking-console-plugin-cert" not found Jul 13 22:00:28.434348 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.434324 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl"] Jul 13 22:00:28.436522 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.436503 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl" Jul 13 22:00:28.438733 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.438699 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 13 22:00:28.439078 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.439050 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 13 22:00:28.439842 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.439825 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qfs9h\"" Jul 13 22:00:28.458007 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.457981 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl"] Jul 13 22:00:28.476539 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.476508 2570 generic.go:356] "Generic (PLEG): container finished" podID="b9418498-5970-4196-a734-4f64df7d4168" containerID="4ca67f076d812b708ce6ca3caca3938dff274509289cde06f7c34909e24be12a" exitCode=0 Jul 13 22:00:28.476640 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.476603 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p4sz" event={"ID":"b9418498-5970-4196-a734-4f64df7d4168","Type":"ContainerDied","Data":"4ca67f076d812b708ce6ca3caca3938dff274509289cde06f7c34909e24be12a"} Jul 13 22:00:28.478150 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.478122 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6b8db" event={"ID":"bf916935-4397-4211-a340-bdaa676732eb","Type":"ContainerStarted","Data":"c64d31ccd9dfa5ce06e12beafd0e4cac13626dff9127415021ffbd6109ad81dc"} Jul 13 22:00:28.524037 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.524008 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:28.526996 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.526975 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbq2h\" (UniqueName: \"kubernetes.io/projected/b29b53f5-6c35-49a8-9c33-9619b64a9811-kube-api-access-mbq2h\") pod \"migrator-7f59b54f6c-vh2hl\" (UID: \"b29b53f5-6c35-49a8-9c33-9619b64a9811\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl" Jul 13 22:00:28.615467 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.615435 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:28.628444 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.628417 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbq2h\" (UniqueName: \"kubernetes.io/projected/b29b53f5-6c35-49a8-9c33-9619b64a9811-kube-api-access-mbq2h\") pod \"migrator-7f59b54f6c-vh2hl\" (UID: \"b29b53f5-6c35-49a8-9c33-9619b64a9811\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl" Jul 13 22:00:28.638004 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.637813 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbq2h\" (UniqueName: \"kubernetes.io/projected/b29b53f5-6c35-49a8-9c33-9619b64a9811-kube-api-access-mbq2h\") pod \"migrator-7f59b54f6c-vh2hl\" (UID: \"b29b53f5-6c35-49a8-9c33-9619b64a9811\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl" Jul 13 22:00:28.744884 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.744837 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-srjs9"] Jul 13 22:00:28.747747 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.747716 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl" Jul 13 22:00:28.751394 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.751374 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.753314 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.753291 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 22:00:28.753314 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.753307 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 22:00:28.753314 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.753365 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xvmjd\"" Jul 13 22:00:28.753314 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.753291 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 22:00:28.753672 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.753654 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 22:00:28.759263 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.759246 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-srjs9"] Jul 13 22:00:28.782638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.782415 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:28.830516 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.830031 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-data-volume\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.830516 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.830074 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.830516 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.830160 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-crio-socket\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.830516 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.830204 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.830516 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.830231 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ghtx\" (UniqueName: \"kubernetes.io/projected/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-api-access-7ghtx\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.858058 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.858023 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:28.917453 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.917385 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl"] Jul 13 22:00:28.920551 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:00:28.920517 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb29b53f5_6c35_49a8_9c33_9619b64a9811.slice/crio-23bed3eebefc9ee7f999ee529d81054c41dbad58e8169028b5248f56db851fd7 WatchSource:0}: Error finding container 23bed3eebefc9ee7f999ee529d81054c41dbad58e8169028b5248f56db851fd7: Status 404 returned error can't find the container with id 23bed3eebefc9ee7f999ee529d81054c41dbad58e8169028b5248f56db851fd7 Jul 13 22:00:28.931255 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.931236 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-data-volume\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.931336 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.931264 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.931404 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:28.931383 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 22:00:28.931455 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:28.931424 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls podName:99a7c1d6-e6d3-4869-a624-6eabb61e9da3 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:29.431410407 +0000 UTC m=+38.614254620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srjs9" (UID: "99a7c1d6-e6d3-4869-a624-6eabb61e9da3") : secret "insights-runtime-extractor-tls" not found Jul 13 22:00:28.931512 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.931461 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-crio-socket\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.931564 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.931516 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.931564 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.931550 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ghtx\" (UniqueName: \"kubernetes.io/projected/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-api-access-7ghtx\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.931669 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.931607 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-data-volume\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.931669 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:28.931624 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 22:00:28.931745 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:28.931689 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-rbac-proxy-cm podName:99a7c1d6-e6d3-4869-a624-6eabb61e9da3 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:29.431672082 +0000 UTC m=+38.614516312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-srjs9" (UID: "99a7c1d6-e6d3-4869-a624-6eabb61e9da3") : configmap "kube-rbac-proxy" not found Jul 13 22:00:28.931745 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.931689 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-crio-socket\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:28.939963 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:28.939941 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ghtx\" (UniqueName: \"kubernetes.io/projected/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-api-access-7ghtx\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:29.051348 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.051312 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:29.116835 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.116801 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:29.277659 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.277571 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:29.370191 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.370159 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:29.436202 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.436162 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:29.436887 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.436225 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:29.436887 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.436269 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:29.436887 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:29.436319 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 22:00:29.436887 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:29.436386 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 22:00:29.436887 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:29.436405 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 22:00:29.436887 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:29.436439 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-rbac-proxy-cm podName:99a7c1d6-e6d3-4869-a624-6eabb61e9da3 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:30.436400343 +0000 UTC m=+39.619244572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-srjs9" (UID: "99a7c1d6-e6d3-4869-a624-6eabb61e9da3") : configmap "kube-rbac-proxy" not found Jul 13 22:00:29.436887 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:29.436456 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert podName:bec934ce-60f6-47d0-8e29-c4ac675f1225 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:31.436448853 +0000 UTC m=+40.619293062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9sg29" (UID: "bec934ce-60f6-47d0-8e29-c4ac675f1225") : secret "networking-console-plugin-cert" not found Jul 13 22:00:29.436887 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:29.436466 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls podName:99a7c1d6-e6d3-4869-a624-6eabb61e9da3 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:30.436460982 +0000 UTC m=+39.619305190 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srjs9" (UID: "99a7c1d6-e6d3-4869-a624-6eabb61e9da3") : secret "insights-runtime-extractor-tls" not found Jul 13 22:00:29.484560 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.484518 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl" event={"ID":"b29b53f5-6c35-49a8-9c33-9619b64a9811","Type":"ContainerStarted","Data":"23bed3eebefc9ee7f999ee529d81054c41dbad58e8169028b5248f56db851fd7"} Jul 13 22:00:29.488651 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.488622 2570 generic.go:356] "Generic (PLEG): container finished" podID="b9418498-5970-4196-a734-4f64df7d4168" containerID="ef5711465ff31f086edcefb1309c82bde86a93d76ffea9d2c9baa1f855931817" exitCode=0 Jul 13 22:00:29.488788 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.488679 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p4sz" event={"ID":"b9418498-5970-4196-a734-4f64df7d4168","Type":"ContainerDied","Data":"ef5711465ff31f086edcefb1309c82bde86a93d76ffea9d2c9baa1f855931817"} Jul 13 22:00:29.641822 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.641790 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:29.704724 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.704687 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:29.739545 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.739510 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:29.739712 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.739600 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:29.739712 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:29.739673 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 22:00:29.739823 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:29.739743 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls podName:0ca89863-2ae9-49cb-8909-abbc0a5b6bb1 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:33.739726442 +0000 UTC m=+42.922570674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls") pod "dns-default-qsg7t" (UID: "0ca89863-2ae9-49cb-8909-abbc0a5b6bb1") : secret "dns-default-metrics-tls" not found Jul 13 22:00:29.739823 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:29.739774 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 22:00:29.739823 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:29.739792 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85ccff9ddd-tqlj9: secret "image-registry-tls" not found Jul 13 22:00:29.739937 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:29.739873 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls podName:7e70b328-f83e-4dec-bb26-b5caab12b0fe nodeName:}" failed. No retries permitted until 2026-07-13 22:00:33.739836748 +0000 UTC m=+42.922680970 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls") pod "image-registry-85ccff9ddd-tqlj9" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe") : secret "image-registry-tls" not found Jul 13 22:00:29.843487 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:29.843406 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:30.445699 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:30.445659 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:30.446393 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:30.445754 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:30.446393 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:30.445913 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 22:00:30.446393 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:30.445984 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls podName:99a7c1d6-e6d3-4869-a624-6eabb61e9da3 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:32.445967511 +0000 UTC m=+41.628811725 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srjs9" (UID: "99a7c1d6-e6d3-4869-a624-6eabb61e9da3") : secret "insights-runtime-extractor-tls" not found Jul 13 22:00:30.446393 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:30.446326 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:30.495906 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:30.495870 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p4sz" event={"ID":"b9418498-5970-4196-a734-4f64df7d4168","Type":"ContainerStarted","Data":"b3b4e63aef842400d1d1287b9999b9ef346317cf8e4d6316b3224b9e15feef7b"} Jul 13 22:00:30.518337 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:30.518278 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9p4sz" podStartSLOduration=9.745829881 podStartE2EDuration="39.518256927s" totalImagesPullingTime="29.772427046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:30.517370254 +0000 UTC m=+39.700214488" watchObservedRunningTime="2026-07-13 22:00:30.518256927 +0000 UTC m=+39.701101159" Jul 13 22:00:30.640966 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:30.640837 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:30.708378 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:30.708342 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:30.935818 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:30.935746 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:31.138884 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.138716 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c9t96_8c06f173-22eb-4265-92c2-45838d12f578/dns-node-resolver/0.log" Jul 13 22:00:31.202936 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.202816 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:31.304308 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.304277 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:31.456710 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.456647 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:31.457008 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:31.456774 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 22:00:31.457008 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:31.456834 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert podName:bec934ce-60f6-47d0-8e29-c4ac675f1225 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:35.45681923 +0000 UTC m=+44.639663440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9sg29" (UID: "bec934ce-60f6-47d0-8e29-c4ac675f1225") : secret "networking-console-plugin-cert" not found Jul 13 22:00:31.501650 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.501607 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl" event={"ID":"b29b53f5-6c35-49a8-9c33-9619b64a9811","Type":"ContainerStarted","Data":"bdae314cba6e81449977a349743d9a0cfdea15063398441264adc7b01f0c2cc7"} Jul 13 22:00:31.501803 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.501710 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl" event={"ID":"b29b53f5-6c35-49a8-9c33-9619b64a9811","Type":"ContainerStarted","Data":"41661da68f73389a374ec20525a49d931a43457104a7167f78bac218d085b0e9"} Jul 13 22:00:31.503787 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.503756 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6b8db" event={"ID":"bf916935-4397-4211-a340-bdaa676732eb","Type":"ContainerStarted","Data":"1b430c47660e472324971ff03c679ed0053d58147548e58cb2b2c9e093d442c8"} Jul 13 22:00:31.517296 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.517248 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vh2hl" podStartSLOduration=1.191581144 podStartE2EDuration="3.517234663s" totalImagesPullingTime="2.325653519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:31.516335773 +0000 UTC m=+40.699180007" watchObservedRunningTime="2026-07-13 22:00:31.517234663 +0000 UTC m=+40.700078895" Jul 13 22:00:31.530503 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.530456 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6b8db" podStartSLOduration=2.7853781250000003 podStartE2EDuration="6.530438755s" totalImagesPullingTime="3.74506063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:31.529980418 +0000 UTC m=+40.712824673" watchObservedRunningTime="2026-07-13 22:00:31.530438755 +0000 UTC m=+40.713282987" Jul 13 22:00:31.624372 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.624334 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:31.725237 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.725158 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:31.977692 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:31.977611 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:32.339144 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:32.339069 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nvk85_4fd817a9-f4c3-443d-a8f7-02820f6c426f/node-ca/0.log" Jul 13 22:00:32.465630 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:32.465593 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:32.466022 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:32.465710 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 22:00:32.466022 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:32.465764 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls podName:99a7c1d6-e6d3-4869-a624-6eabb61e9da3 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:36.465749911 +0000 UTC m=+45.648594120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srjs9" (UID: "99a7c1d6-e6d3-4869-a624-6eabb61e9da3") : secret "insights-runtime-extractor-tls" not found Jul 13 22:00:32.739592 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:32.739559 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6b8db_bf916935-4397-4211-a340-bdaa676732eb/serve-healthcheck-canary/0.log" Jul 13 22:00:33.149123 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:33.149091 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-vh2hl_b29b53f5-6c35-49a8-9c33-9619b64a9811/migrator/0.log" Jul 13 22:00:33.339351 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:33.339326 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-vh2hl_b29b53f5-6c35-49a8-9c33-9619b64a9811/graceful-termination/0.log" Jul 13 22:00:33.776105 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:33.776069 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:33.776465 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:33.776160 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:33.776465 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:33.776214 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 22:00:33.776465 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:33.776246 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 22:00:33.776465 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:33.776254 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85ccff9ddd-tqlj9: secret "image-registry-tls" not found Jul 13 22:00:33.776465 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:33.776298 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls podName:0ca89863-2ae9-49cb-8909-abbc0a5b6bb1 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:41.77628617 +0000 UTC m=+50.959130380 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls") pod "dns-default-qsg7t" (UID: "0ca89863-2ae9-49cb-8909-abbc0a5b6bb1") : secret "dns-default-metrics-tls" not found Jul 13 22:00:33.776465 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:33.776313 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls podName:7e70b328-f83e-4dec-bb26-b5caab12b0fe nodeName:}" failed. No retries permitted until 2026-07-13 22:00:41.776305033 +0000 UTC m=+50.959149242 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls") pod "image-registry-85ccff9ddd-tqlj9" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe") : secret "image-registry-tls" not found Jul 13 22:00:34.748546 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:34.748510 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:34.801151 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:34.801121 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:35.367564 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:35.367528 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:35.488886 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:35.488839 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:35.489025 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:35.489002 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 22:00:35.489082 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:35.489072 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert podName:bec934ce-60f6-47d0-8e29-c4ac675f1225 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:43.489056227 +0000 UTC m=+52.671900437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9sg29" (UID: "bec934ce-60f6-47d0-8e29-c4ac675f1225") : secret "networking-console-plugin-cert" not found Jul 13 22:00:36.174104 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:36.174059 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:36.282017 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:36.281978 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:36.390645 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:36.390604 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:36.492501 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:36.492418 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:36.496467 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:36.496443 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:36.496585 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:36.496571 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 22:00:36.496652 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:36.496641 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls podName:99a7c1d6-e6d3-4869-a624-6eabb61e9da3 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:44.496623356 +0000 UTC m=+53.679467579 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srjs9" (UID: "99a7c1d6-e6d3-4869-a624-6eabb61e9da3") : secret "insights-runtime-extractor-tls" not found Jul 13 22:00:36.599484 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:36.599456 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:37.060915 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:37.060882 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:37.659423 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:37.659386 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:37.967231 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:37.967154 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:38.069047 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:38.069013 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:38.170751 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:38.170716 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:38.318655 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:38.318566 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:38.675375 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:38.675345 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:39.467586 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:39.467552 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:39.867837 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:39.867808 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:39.994003 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:39.993971 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:40.095209 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:40.095177 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:40.267593 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:40.267506 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:40.458538 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:40.458495 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:40.563025 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:40.562942 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:41.653339 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:41.653300 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:41.843805 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:41.843766 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:41.843992 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:41.843819 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:41.846672 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:41.846644 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls\") pod \"image-registry-85ccff9ddd-tqlj9\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:41.846774 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:41.846642 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ca89863-2ae9-49cb-8909-abbc0a5b6bb1-metrics-tls\") pod \"dns-default-qsg7t\" (UID: \"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1\") " pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:41.852459 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:41.852430 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:41.913550 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:41.913478 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:41.928104 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:41.928080 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:42.053162 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:42.053070 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:42.057837 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:42.057816 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85ccff9ddd-tqlj9"] Jul 13 22:00:42.060624 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:00:42.060596 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e70b328_f83e_4dec_bb26_b5caab12b0fe.slice/crio-a3b9e57389416c7049cdffe995a9723d3225e1201f640c5a6700b139f6b3b6e4 WatchSource:0}: Error finding container a3b9e57389416c7049cdffe995a9723d3225e1201f640c5a6700b139f6b3b6e4: Status 404 returned error can't find the container with id a3b9e57389416c7049cdffe995a9723d3225e1201f640c5a6700b139f6b3b6e4 Jul 13 22:00:42.069617 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:42.069593 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qsg7t"] Jul 13 22:00:42.072188 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:00:42.072164 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ca89863_2ae9_49cb_8909_abbc0a5b6bb1.slice/crio-7ed7113cd3775df97d70ef8c86289182386cb15a1a2ca41a9a156ed4da93f6e1 WatchSource:0}: Error finding container 7ed7113cd3775df97d70ef8c86289182386cb15a1a2ca41a9a156ed4da93f6e1: Status 404 returned error can't find the container with id 7ed7113cd3775df97d70ef8c86289182386cb15a1a2ca41a9a156ed4da93f6e1 Jul 13 22:00:42.269330 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:42.269226 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:42.532691 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:42.532607 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qsg7t" event={"ID":"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1","Type":"ContainerStarted","Data":"7ed7113cd3775df97d70ef8c86289182386cb15a1a2ca41a9a156ed4da93f6e1"} Jul 13 22:00:42.534086 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:42.534059 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" event={"ID":"7e70b328-f83e-4dec-bb26-b5caab12b0fe","Type":"ContainerStarted","Data":"a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f"} Jul 13 22:00:42.534219 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:42.534106 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" event={"ID":"7e70b328-f83e-4dec-bb26-b5caab12b0fe","Type":"ContainerStarted","Data":"a3b9e57389416c7049cdffe995a9723d3225e1201f640c5a6700b139f6b3b6e4"} Jul 13 22:00:42.534219 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:42.534196 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:00:42.555198 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:42.555149 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" podStartSLOduration=23.555132218 podStartE2EDuration="23.555132218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:42.554368933 +0000 UTC m=+51.737213191" watchObservedRunningTime="2026-07-13 22:00:42.555132218 +0000 UTC m=+51.737976448" Jul 13 22:00:42.747659 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:42.747622 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:42.804602 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:42.804526 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:43.559670 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:43.559627 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:43.559885 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:43.559822 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 22:00:43.559953 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:00:43.559899 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert podName:bec934ce-60f6-47d0-8e29-c4ac675f1225 nodeName:}" failed. No retries permitted until 2026-07-13 22:00:59.559880145 +0000 UTC m=+68.742724355 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-9sg29" (UID: "bec934ce-60f6-47d0-8e29-c4ac675f1225") : secret "networking-console-plugin-cert" not found Jul 13 22:00:43.656285 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:43.656247 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:43.863720 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:43.863687 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:43.929864 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:43.929825 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:44.066162 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:44.066124 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:44.259945 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:44.259842 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:44.462166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:44.462125 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:44.541450 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:44.541417 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qsg7t" event={"ID":"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1","Type":"ContainerStarted","Data":"73b74243811cffb8ca499d98c9bdf1c5fafe4326dafe2e75ce327794186f179a"} Jul 13 22:00:44.569100 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:44.569076 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:44.571128 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:44.571111 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99a7c1d6-e6d3-4869-a624-6eabb61e9da3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srjs9\" (UID: \"99a7c1d6-e6d3-4869-a624-6eabb61e9da3\") " pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:44.663042 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:44.663010 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-srjs9" Jul 13 22:00:44.794828 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:44.794755 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-srjs9"] Jul 13 22:00:44.797477 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:00:44.797454 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a7c1d6_e6d3_4869_a624_6eabb61e9da3.slice/crio-29dc297caaaa7d89ba30cb430daad1ca5abee285ae8eb7864e123ba50135eac9 WatchSource:0}: Error finding container 29dc297caaaa7d89ba30cb430daad1ca5abee285ae8eb7864e123ba50135eac9: Status 404 returned error can't find the container with id 29dc297caaaa7d89ba30cb430daad1ca5abee285ae8eb7864e123ba50135eac9 Jul 13 22:00:45.459463 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:45.459401 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:45.546697 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:45.546657 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srjs9" event={"ID":"99a7c1d6-e6d3-4869-a624-6eabb61e9da3","Type":"ContainerStarted","Data":"29dc297caaaa7d89ba30cb430daad1ca5abee285ae8eb7864e123ba50135eac9"} Jul 13 22:00:45.549326 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:45.549298 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qsg7t" event={"ID":"0ca89863-2ae9-49cb-8909-abbc0a5b6bb1","Type":"ContainerStarted","Data":"578fe322f3819f7a4ee6a0a6e0e205564d39849026a8dd768270e78d0ccdb788"} Jul 13 22:00:45.550114 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:45.550094 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:45.567735 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:45.567678 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qsg7t" podStartSLOduration=18.278401609 podStartE2EDuration="20.567658236s" totalImagesPullingTime="2.289256627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:45.56683409 +0000 UTC m=+54.749678343" watchObservedRunningTime="2026-07-13 22:00:45.567658236 +0000 UTC m=+54.750502448" Jul 13 22:00:45.659337 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:45.659306 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:46.057685 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:46.057649 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:46.259393 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:46.259354 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:46.461660 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:46.461616 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:46.855403 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:46.855304 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:48.561464 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:48.561429 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srjs9" event={"ID":"99a7c1d6-e6d3-4869-a624-6eabb61e9da3","Type":"ContainerStarted","Data":"dc7aafa0b1134c9abfd0fad7503d380ec937406092ac8a05a9869db170b18dec"} Jul 13 22:00:48.760194 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:48.760103 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:48.852040 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:48.852007 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:48.962437 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:48.962379 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:49.069909 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:49.069791 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:49.267638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:49.267589 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:49.445695 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:49.445664 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:49.522100 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:49.522067 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:49.565801 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:49.565771 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srjs9" event={"ID":"99a7c1d6-e6d3-4869-a624-6eabb61e9da3","Type":"ContainerStarted","Data":"c879ee403937b47fa2611f694c490efdaeb29cf6ad32a469fa6886e9f16b1f86"} Jul 13 22:00:49.631456 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:49.631419 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:49.689590 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:49.689554 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:49.820535 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:49.820441 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:49.925059 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:49.925027 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:50.289844 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:50.289790 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:50.834511 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:50.834478 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:50.986715 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:50.986663 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg"] Jul 13 22:00:51.012151 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.012114 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:51.017293 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.017266 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg"] Jul 13 22:00:51.017436 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.017375 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:51.017862 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.017758 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5715bf14-b198-42d5-a2dc-b59a77dfe381-klusterlet-config\") pod \"klusterlet-addon-workmgr-59575744bc-4cspg\" (UID: \"5715bf14-b198-42d5-a2dc-b59a77dfe381\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:51.017862 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.017796 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d7mz\" (UniqueName: \"kubernetes.io/projected/5715bf14-b198-42d5-a2dc-b59a77dfe381-kube-api-access-6d7mz\") pod \"klusterlet-addon-workmgr-59575744bc-4cspg\" (UID: \"5715bf14-b198-42d5-a2dc-b59a77dfe381\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:51.018020 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.017872 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5715bf14-b198-42d5-a2dc-b59a77dfe381-tmp\") pod \"klusterlet-addon-workmgr-59575744bc-4cspg\" (UID: \"5715bf14-b198-42d5-a2dc-b59a77dfe381\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:51.020813 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.020729 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 13 22:00:51.020813 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.020732 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 13 22:00:51.021380 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.021360 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 13 22:00:51.021487 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.021396 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 13 22:00:51.086098 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.086016 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-2cfgx"] Jul 13 22:00:51.118771 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.118741 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5715bf14-b198-42d5-a2dc-b59a77dfe381-klusterlet-config\") pod \"klusterlet-addon-workmgr-59575744bc-4cspg\" (UID: \"5715bf14-b198-42d5-a2dc-b59a77dfe381\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:51.118962 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.118891 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d7mz\" (UniqueName: \"kubernetes.io/projected/5715bf14-b198-42d5-a2dc-b59a77dfe381-kube-api-access-6d7mz\") pod \"klusterlet-addon-workmgr-59575744bc-4cspg\" (UID: \"5715bf14-b198-42d5-a2dc-b59a77dfe381\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:51.118962 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.118957 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5715bf14-b198-42d5-a2dc-b59a77dfe381-tmp\") pod \"klusterlet-addon-workmgr-59575744bc-4cspg\" (UID: \"5715bf14-b198-42d5-a2dc-b59a77dfe381\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:51.119343 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.119316 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5715bf14-b198-42d5-a2dc-b59a77dfe381-tmp\") pod \"klusterlet-addon-workmgr-59575744bc-4cspg\" (UID: \"5715bf14-b198-42d5-a2dc-b59a77dfe381\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:51.121583 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.121560 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5715bf14-b198-42d5-a2dc-b59a77dfe381-klusterlet-config\") pod \"klusterlet-addon-workmgr-59575744bc-4cspg\" (UID: \"5715bf14-b198-42d5-a2dc-b59a77dfe381\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:51.129199 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.129179 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85ccff9ddd-tqlj9"] Jul 13 22:00:51.129199 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.129192 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-2cfgx" Jul 13 22:00:51.129363 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.129234 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-2cfgx"] Jul 13 22:00:51.131422 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.131368 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d7mz\" (UniqueName: \"kubernetes.io/projected/5715bf14-b198-42d5-a2dc-b59a77dfe381-kube-api-access-6d7mz\") pod \"klusterlet-addon-workmgr-59575744bc-4cspg\" (UID: \"5715bf14-b198-42d5-a2dc-b59a77dfe381\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:51.131422 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.131382 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 22:00:51.131798 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.131780 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mctpk\"" Jul 13 22:00:51.131892 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.131808 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 22:00:51.196720 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.196684 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d77b9978f-h64zc"] Jul 13 22:00:51.212744 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.212693 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:51.219585 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.219562 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw8v4\" (UniqueName: \"kubernetes.io/projected/0b9017b7-7904-4479-bde7-bf0075a083c4-kube-api-access-bw8v4\") pod \"downloads-68c9cbb497-2cfgx\" (UID: \"0b9017b7-7904-4479-bde7-bf0075a083c4\") " pod="openshift-console/downloads-68c9cbb497-2cfgx" Jul 13 22:00:51.235718 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.235694 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d77b9978f-h64zc"] Jul 13 22:00:51.235900 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.235815 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.318345 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.318290 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:51.320033 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.320010 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e940fa0-f70d-472a-947b-e85e4ecd07c6-trusted-ca\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.320167 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.320040 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e940fa0-f70d-472a-947b-e85e4ecd07c6-registry-certificates\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.320167 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.320058 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e940fa0-f70d-472a-947b-e85e4ecd07c6-bound-sa-token\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.320167 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.320146 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e940fa0-f70d-472a-947b-e85e4ecd07c6-installation-pull-secrets\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.320294 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.320231 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e940fa0-f70d-472a-947b-e85e4ecd07c6-registry-tls\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.320294 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.320266 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e940fa0-f70d-472a-947b-e85e4ecd07c6-image-registry-private-configuration\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.320383 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.320311 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e940fa0-f70d-472a-947b-e85e4ecd07c6-ca-trust-extracted\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.320383 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.320343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw8v4\" (UniqueName: \"kubernetes.io/projected/0b9017b7-7904-4479-bde7-bf0075a083c4-kube-api-access-bw8v4\") pod \"downloads-68c9cbb497-2cfgx\" (UID: \"0b9017b7-7904-4479-bde7-bf0075a083c4\") " pod="openshift-console/downloads-68c9cbb497-2cfgx" Jul 13 22:00:51.320475 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.320410 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcx2p\" (UniqueName: \"kubernetes.io/projected/4e940fa0-f70d-472a-947b-e85e4ecd07c6-kube-api-access-mcx2p\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.330339 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.330311 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 13 22:00:51.335075 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.335054 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 22:00:51.338666 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.338605 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:51.346136 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.346116 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 22:00:51.357759 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.357733 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw8v4\" (UniqueName: \"kubernetes.io/projected/0b9017b7-7904-4479-bde7-bf0075a083c4-kube-api-access-bw8v4\") pod \"downloads-68c9cbb497-2cfgx\" (UID: \"0b9017b7-7904-4479-bde7-bf0075a083c4\") " pod="openshift-console/downloads-68c9cbb497-2cfgx" Jul 13 22:00:51.411747 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.411703 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:51.421275 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.421247 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcx2p\" (UniqueName: \"kubernetes.io/projected/4e940fa0-f70d-472a-947b-e85e4ecd07c6-kube-api-access-mcx2p\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.421431 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.421300 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e940fa0-f70d-472a-947b-e85e4ecd07c6-trusted-ca\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.421431 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.421329 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e940fa0-f70d-472a-947b-e85e4ecd07c6-registry-certificates\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.421431 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.421351 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e940fa0-f70d-472a-947b-e85e4ecd07c6-bound-sa-token\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.421431 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.421375 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e940fa0-f70d-472a-947b-e85e4ecd07c6-installation-pull-secrets\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.421652 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.421514 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e940fa0-f70d-472a-947b-e85e4ecd07c6-registry-tls\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.421652 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.421569 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e940fa0-f70d-472a-947b-e85e4ecd07c6-image-registry-private-configuration\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.421652 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.421616 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e940fa0-f70d-472a-947b-e85e4ecd07c6-ca-trust-extracted\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.422113 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.422081 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e940fa0-f70d-472a-947b-e85e4ecd07c6-ca-trust-extracted\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.422364 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.422338 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e940fa0-f70d-472a-947b-e85e4ecd07c6-registry-certificates\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.422470 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.422391 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e940fa0-f70d-472a-947b-e85e4ecd07c6-trusted-ca\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.424667 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.424635 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e940fa0-f70d-472a-947b-e85e4ecd07c6-registry-tls\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.424899 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.424873 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e940fa0-f70d-472a-947b-e85e4ecd07c6-image-registry-private-configuration\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.424899 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.424891 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e940fa0-f70d-472a-947b-e85e4ecd07c6-installation-pull-secrets\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.434453 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.434425 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcx2p\" (UniqueName: \"kubernetes.io/projected/4e940fa0-f70d-472a-947b-e85e4ecd07c6-kube-api-access-mcx2p\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.434818 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.434792 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e940fa0-f70d-472a-947b-e85e4ecd07c6-bound-sa-token\") pod \"image-registry-5d77b9978f-h64zc\" (UID: \"4e940fa0-f70d-472a-947b-e85e4ecd07c6\") " pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:51.448977 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.448947 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mctpk\"" Jul 13 22:00:51.457537 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.457507 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-2cfgx" Jul 13 22:00:51.510111 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.510071 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:51.551127 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:51.551096 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:52.064927 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.064896 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d77b9978f-h64zc"] Jul 13 22:00:52.092542 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:00:52.092510 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e940fa0_f70d_472a_947b_e85e4ecd07c6.slice/crio-11ad813b599fbb9ac3f52250d4062a4435fd55a7d887eb67706bc096fe10a604 WatchSource:0}: Error finding container 11ad813b599fbb9ac3f52250d4062a4435fd55a7d887eb67706bc096fe10a604: Status 404 returned error can't find the container with id 11ad813b599fbb9ac3f52250d4062a4435fd55a7d887eb67706bc096fe10a604 Jul 13 22:00:52.234111 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.233958 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg"] Jul 13 22:00:52.236710 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:00:52.236666 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5715bf14_b198_42d5_a2dc_b59a77dfe381.slice/crio-2e0e901fb14845a46494fd3f3a81180aaa48d9ad7da0b72de70a40d55d9edaa4 WatchSource:0}: Error finding container 2e0e901fb14845a46494fd3f3a81180aaa48d9ad7da0b72de70a40d55d9edaa4: Status 404 returned error can't find the container with id 2e0e901fb14845a46494fd3f3a81180aaa48d9ad7da0b72de70a40d55d9edaa4 Jul 13 22:00:52.239986 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.239969 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-2cfgx"] Jul 13 22:00:52.240409 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:00:52.240390 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9017b7_7904_4479_bde7_bf0075a083c4.slice/crio-614698721f8837b845b580cbe8a1b154044369d3bdb274667e84896b35caa228 WatchSource:0}: Error finding container 614698721f8837b845b580cbe8a1b154044369d3bdb274667e84896b35caa228: Status 404 returned error can't find the container with id 614698721f8837b845b580cbe8a1b154044369d3bdb274667e84896b35caa228 Jul 13 22:00:52.467729 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.467663 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9grcb" Jul 13 22:00:52.577120 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.577086 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" event={"ID":"5715bf14-b198-42d5-a2dc-b59a77dfe381","Type":"ContainerStarted","Data":"2e0e901fb14845a46494fd3f3a81180aaa48d9ad7da0b72de70a40d55d9edaa4"} Jul 13 22:00:52.579049 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.579021 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srjs9" event={"ID":"99a7c1d6-e6d3-4869-a624-6eabb61e9da3","Type":"ContainerStarted","Data":"9e9af4af85f5debdc74dfce9f934f02e60945cdb76decab0c171219fcd7de65b"} Jul 13 22:00:52.580119 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.580091 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-2cfgx" event={"ID":"0b9017b7-7904-4479-bde7-bf0075a083c4","Type":"ContainerStarted","Data":"614698721f8837b845b580cbe8a1b154044369d3bdb274667e84896b35caa228"} Jul 13 22:00:52.581249 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.581225 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:52.581670 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.581652 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" event={"ID":"4e940fa0-f70d-472a-947b-e85e4ecd07c6","Type":"ContainerStarted","Data":"ac1a230307f426ae1a4df811b03536e7e690de3316e2b00043a32cbc64a82afd"} Jul 13 22:00:52.581734 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.581685 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" event={"ID":"4e940fa0-f70d-472a-947b-e85e4ecd07c6","Type":"ContainerStarted","Data":"11ad813b599fbb9ac3f52250d4062a4435fd55a7d887eb67706bc096fe10a604"} Jul 13 22:00:52.581841 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.581830 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:00:52.600340 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.600289 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-srjs9" podStartSLOduration=17.397090839 podStartE2EDuration="24.600271968s" totalImagesPullingTime="7.203181129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 22:00:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:52.599256699 +0000 UTC m=+61.782100931" watchObservedRunningTime="2026-07-13 22:00:52.600271968 +0000 UTC m=+61.783116200" Jul 13 22:00:52.676301 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.676258 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:52.779616 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:52.779522 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:53.886002 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:53.885963 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:53.968127 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:53.968055 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:55.555692 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:55.555651 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:55.628529 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:55.628497 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:56.378771 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:56.378736 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:56.483330 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:56.483274 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:56.558473 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:56.558444 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qsg7t" Jul 13 22:00:56.584184 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:56.584133 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:56.587256 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:56.587213 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" podStartSLOduration=5.587196731 podStartE2EDuration="5.587196731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:00:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:52.630794012 +0000 UTC m=+61.813638244" watchObservedRunningTime="2026-07-13 22:00:56.587196731 +0000 UTC m=+65.770040963" Jul 13 22:00:56.853033 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:56.852994 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:57.168716 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.168674 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:57.170940 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.170910 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 22:00:57.182248 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.182220 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a422d89-db30-488c-8a59-7282e92cd6ab-metrics-certs\") pod \"network-metrics-daemon-l42gj\" (UID: \"3a422d89-db30-488c-8a59-7282e92cd6ab\") " pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:57.224536 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.224505 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b6tsw\"" Jul 13 22:00:57.232272 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.232248 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l42gj" Jul 13 22:00:57.271616 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.271280 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz9qn\" (UniqueName: \"kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn\") pod \"network-check-target-rnlqj\" (UID: \"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad\") " pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:57.273483 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.273457 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 22:00:57.284245 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.284219 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 22:00:57.297562 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.297537 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz9qn\" (UniqueName: \"kubernetes.io/projected/3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad-kube-api-access-pz9qn\") pod \"network-check-target-rnlqj\" (UID: \"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad\") " pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:57.323479 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.323454 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gmvbg\"" Jul 13 22:00:57.332464 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.332430 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:00:57.383299 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.383222 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l42gj"] Jul 13 22:00:57.387399 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:00:57.387365 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a422d89_db30_488c_8a59_7282e92cd6ab.slice/crio-f75dcebdb0f2c534d23064a735a0dbd313dc95360684a2cc18419449858050bc WatchSource:0}: Error finding container f75dcebdb0f2c534d23064a735a0dbd313dc95360684a2cc18419449858050bc: Status 404 returned error can't find the container with id f75dcebdb0f2c534d23064a735a0dbd313dc95360684a2cc18419449858050bc Jul 13 22:00:57.473814 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.473780 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:57.475910 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.475884 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 22:00:57.483453 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.483427 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rnlqj"] Jul 13 22:00:57.487381 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.487359 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a11f9f29-be8e-4926-af3d-f2d4ed6b00d6-original-pull-secret\") pod \"global-pull-secret-syncer-4w92z\" (UID: \"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6\") " pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:57.487506 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:00:57.487470 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3576c7e9_760d_4f8f_ba92_4e3d2ebbb4ad.slice/crio-3e604555e5f663133008092ed701185d300b93b66249b0f0da8d457f297fd40c WatchSource:0}: Error finding container 3e604555e5f663133008092ed701185d300b93b66249b0f0da8d457f297fd40c: Status 404 returned error can't find the container with id 3e604555e5f663133008092ed701185d300b93b66249b0f0da8d457f297fd40c Jul 13 22:00:57.528326 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.528295 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92z" Jul 13 22:00:57.600985 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.600958 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rnlqj" event={"ID":"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad","Type":"ContainerStarted","Data":"3e604555e5f663133008092ed701185d300b93b66249b0f0da8d457f297fd40c"} Jul 13 22:00:57.602166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.602111 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l42gj" event={"ID":"3a422d89-db30-488c-8a59-7282e92cd6ab","Type":"ContainerStarted","Data":"f75dcebdb0f2c534d23064a735a0dbd313dc95360684a2cc18419449858050bc"} Jul 13 22:00:57.603499 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.603472 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" event={"ID":"5715bf14-b198-42d5-a2dc-b59a77dfe381","Type":"ContainerStarted","Data":"1b9de481c8b827472f42de74614fe969c571050249b30bbb57e42125201e7f1c"} Jul 13 22:00:57.603693 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.603676 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:57.605457 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.605440 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" Jul 13 22:00:57.618306 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.618071 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59575744bc-4cspg" podStartSLOduration=3.160240197 podStartE2EDuration="7.618053833s" totalImagesPullingTime="4.457813636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:57.617278659 +0000 UTC m=+66.800122889" watchObservedRunningTime="2026-07-13 22:00:57.618053833 +0000 UTC m=+66.800898065" Jul 13 22:00:57.663970 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:57.663935 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4w92z"] Jul 13 22:00:57.666804 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:00:57.666774 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda11f9f29_be8e_4926_af3d_f2d4ed6b00d6.slice/crio-163a72fcc471335dc9050cbdd3bb799ebca6e7af5adc4638fafb6d064dda98d2 WatchSource:0}: Error finding container 163a72fcc471335dc9050cbdd3bb799ebca6e7af5adc4638fafb6d064dda98d2: Status 404 returned error can't find the container with id 163a72fcc471335dc9050cbdd3bb799ebca6e7af5adc4638fafb6d064dda98d2 Jul 13 22:00:58.609201 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:58.609120 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4w92z" event={"ID":"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6","Type":"ContainerStarted","Data":"163a72fcc471335dc9050cbdd3bb799ebca6e7af5adc4638fafb6d064dda98d2"} Jul 13 22:00:59.592287 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:59.592177 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:59.595656 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:59.595605 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bec934ce-60f6-47d0-8e29-c4ac675f1225-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9sg29\" (UID: \"bec934ce-60f6-47d0-8e29-c4ac675f1225\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:00:59.617837 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:59.617763 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l42gj" event={"ID":"3a422d89-db30-488c-8a59-7282e92cd6ab","Type":"ContainerStarted","Data":"534a2636c9325e10d8fd10398fd1faa347b94ae34130fb4c6093ca9aaf8305fd"} Jul 13 22:00:59.617837 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:59.617834 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l42gj" event={"ID":"3a422d89-db30-488c-8a59-7282e92cd6ab","Type":"ContainerStarted","Data":"52a4f4f9c8837d61f389ea74879afcebb8e22e5ed99cc6377e53a7002ab211fa"} Jul 13 22:00:59.636660 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:59.636598 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l42gj" podStartSLOduration=67.163059031 podStartE2EDuration="1m8.636577725s" totalImagesPullingTime="1.473518694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:00:59.634146194 +0000 UTC m=+68.816990427" watchObservedRunningTime="2026-07-13 22:00:59.636577725 +0000 UTC m=+68.819421958" Jul 13 22:00:59.728812 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:59.728745 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:59.829347 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:59.829235 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:59.867134 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:59.867097 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rxnb2\"" Jul 13 22:00:59.875907 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:00:59.875883 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" Jul 13 22:01:00.078416 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.078384 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58f4c96f47-bbdtg"] Jul 13 22:01:00.080598 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.080573 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.083985 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.083961 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 22:01:00.084123 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.083994 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 22:01:00.084236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.084217 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 22:01:00.084519 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.084499 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-crbrs\"" Jul 13 22:01:00.084628 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.084564 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 22:01:00.084628 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.084600 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 22:01:00.092968 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.092823 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 22:01:00.097687 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.097563 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-serving-cert\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.097687 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.097604 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-trusted-ca-bundle\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.097687 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.097638 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5n4p\" (UniqueName: \"kubernetes.io/projected/f4cb6755-6bbf-40c8-883a-f383bdf32de2-kube-api-access-v5n4p\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.097925 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.097697 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-oauth-serving-cert\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.097925 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.097761 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-oauth-config\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.097925 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.097788 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-service-ca\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.097925 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.097822 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-config\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.098511 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.098299 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f4c96f47-bbdtg"] Jul 13 22:01:00.099799 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.099776 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:00.185634 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.185392 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:00.199450 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.199410 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-serving-cert\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.199638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.199455 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-trusted-ca-bundle\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.199638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.199490 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5n4p\" (UniqueName: \"kubernetes.io/projected/f4cb6755-6bbf-40c8-883a-f383bdf32de2-kube-api-access-v5n4p\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.199638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.199521 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-oauth-serving-cert\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.199638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.199572 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-oauth-config\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.199638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.199597 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-service-ca\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.199638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.199627 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-config\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.201040 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.200419 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-config\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.201040 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.200632 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-oauth-serving-cert\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.201040 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.200724 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-trusted-ca-bundle\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.201202 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.201066 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-service-ca\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.203735 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.203697 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-serving-cert\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.204140 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.204122 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-oauth-config\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.209017 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.208986 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5n4p\" (UniqueName: \"kubernetes.io/projected/f4cb6755-6bbf-40c8-883a-f383bdf32de2-kube-api-access-v5n4p\") pod \"console-58f4c96f47-bbdtg\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.393831 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.393794 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:00.607344 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.607032 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:00.682890 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:00.682836 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:01.136791 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:01.136744 2570 patch_prober.go:28] interesting pod/image-registry-85ccff9ddd-tqlj9 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 13 22:01:01.136965 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:01.136809 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" podUID="7e70b328-f83e-4dec-bb26-b5caab12b0fe" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 22:01:01.809011 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:01.808968 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:02.868254 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:02.868229 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-9sg29"] Jul 13 22:01:02.872535 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:01:02.872466 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec934ce_60f6_47d0_8e29_c4ac675f1225.slice/crio-f5fd3e005716bdd1f0508f4992d4c7a2213b168435751fd7b97c56aa14462304 WatchSource:0}: Error finding container f5fd3e005716bdd1f0508f4992d4c7a2213b168435751fd7b97c56aa14462304: Status 404 returned error can't find the container with id f5fd3e005716bdd1f0508f4992d4c7a2213b168435751fd7b97c56aa14462304 Jul 13 22:01:02.895048 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:02.895019 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f4c96f47-bbdtg"] Jul 13 22:01:02.898316 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:01:02.898285 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4cb6755_6bbf_40c8_883a_f383bdf32de2.slice/crio-17be1af0dc4310ebe6e7a9ca08e56bfac7bff522cd393d99e65bae9048cdd65a WatchSource:0}: Error finding container 17be1af0dc4310ebe6e7a9ca08e56bfac7bff522cd393d99e65bae9048cdd65a: Status 404 returned error can't find the container with id 17be1af0dc4310ebe6e7a9ca08e56bfac7bff522cd393d99e65bae9048cdd65a Jul 13 22:01:02.979720 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:02.979625 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:03.054474 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:03.054442 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:03.637571 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:03.637533 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rnlqj" event={"ID":"3576c7e9-760d-4f8f-ba92-4e3d2ebbb4ad","Type":"ContainerStarted","Data":"9c644c513bbbb415146c7a82082e8f401c2e7a52395d536d66d7ce2243c10b81"} Jul 13 22:01:03.637770 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:03.637628 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:01:03.639808 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:03.639776 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4w92z" event={"ID":"a11f9f29-be8e-4926-af3d-f2d4ed6b00d6","Type":"ContainerStarted","Data":"20c30c016d8dc960f7149b3f398ce2e7b5adbc6a416e36b89d6b2236ae7b1064"} Jul 13 22:01:03.641329 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:03.641282 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f4c96f47-bbdtg" event={"ID":"f4cb6755-6bbf-40c8-883a-f383bdf32de2","Type":"ContainerStarted","Data":"17be1af0dc4310ebe6e7a9ca08e56bfac7bff522cd393d99e65bae9048cdd65a"} Jul 13 22:01:03.642543 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:03.642523 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" event={"ID":"bec934ce-60f6-47d0-8e29-c4ac675f1225","Type":"ContainerStarted","Data":"f5fd3e005716bdd1f0508f4992d4c7a2213b168435751fd7b97c56aa14462304"} Jul 13 22:01:03.658035 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:03.657790 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rnlqj" podStartSLOduration=67.43016076 podStartE2EDuration="1m12.657774311s" totalImagesPullingTime="5.227613551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:01:03.656083587 +0000 UTC m=+72.838927823" watchObservedRunningTime="2026-07-13 22:01:03.657774311 +0000 UTC m=+72.840618546" Jul 13 22:01:03.670375 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:03.670321 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4w92z" podStartSLOduration=66.607652798 podStartE2EDuration="1m11.670302957s" totalImagesPullingTime="5.062650159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:01:03.670119121 +0000 UTC m=+72.852963353" watchObservedRunningTime="2026-07-13 22:01:03.670302957 +0000 UTC m=+72.853147189" Jul 13 22:01:04.402881 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:04.402826 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:05.659392 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:05.659344 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:05.761250 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:05.761219 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:05.867005 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:05.866754 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:05.989455 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:05.989368 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:06.010500 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.010465 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm"] Jul 13 22:01:06.013148 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.013128 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.015624 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.015603 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 22:01:06.015751 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.015679 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 13 22:01:06.015751 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.015678 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 13 22:01:06.015751 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.015744 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 22:01:06.016087 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.016066 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-99rzw\"" Jul 13 22:01:06.016183 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.016069 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 22:01:06.027783 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.027763 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm"] Jul 13 22:01:06.031380 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.031358 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-jsfmg"] Jul 13 22:01:06.033553 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.033535 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.036293 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.036272 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 13 22:01:06.037192 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.036841 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-p929p\"" Jul 13 22:01:06.037192 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.036959 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 13 22:01:06.037192 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.037020 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 13 22:01:06.050169 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.050139 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvvfg\" (UniqueName: \"kubernetes.io/projected/e493819d-9f49-421e-a534-9579285072be-kube-api-access-zvvfg\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.050270 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.050183 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e493819d-9f49-421e-a534-9579285072be-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.050270 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.050256 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e493819d-9f49-421e-a534-9579285072be-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.050364 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.050308 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e493819d-9f49-421e-a534-9579285072be-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.053166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.053147 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-jsfmg"] Jul 13 22:01:06.057863 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.057831 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7s9sr"] Jul 13 22:01:06.060162 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.060143 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.062510 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.062486 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 22:01:06.062613 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.062492 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 22:01:06.062613 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.062564 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 22:01:06.062613 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.062577 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k57rd\"" Jul 13 22:01:06.085928 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.085903 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:06.151615 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.151582 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.151615 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.151622 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.151886 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.151656 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e493819d-9f49-421e-a534-9579285072be-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.151886 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.151678 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtpvq\" (UniqueName: \"kubernetes.io/projected/5c88bbb5-bae1-4930-acb2-acac3386a366-kube-api-access-jtpvq\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.151886 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.151708 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.151886 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.151767 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5c88bbb5-bae1-4930-acb2-acac3386a366-root\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.151886 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:01:06.151780 2570 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 13 22:01:06.151886 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.151804 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e493819d-9f49-421e-a534-9579285072be-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.151886 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:01:06.151882 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e493819d-9f49-421e-a534-9579285072be-openshift-state-metrics-tls podName:e493819d-9f49-421e-a534-9579285072be nodeName:}" failed. No retries permitted until 2026-07-13 22:01:06.651838164 +0000 UTC m=+75.834682379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e493819d-9f49-421e-a534-9579285072be-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-mbdmm" (UID: "e493819d-9f49-421e-a534-9579285072be") : secret "openshift-state-metrics-tls" not found Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.151905 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-wtmp\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.151951 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-tls\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.151976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c88bbb5-bae1-4930-acb2-acac3386a366-metrics-client-ca\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.151998 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x87j\" (UniqueName: \"kubernetes.io/projected/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-api-access-4x87j\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.152045 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvvfg\" (UniqueName: \"kubernetes.io/projected/e493819d-9f49-421e-a534-9579285072be-kube-api-access-zvvfg\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.152062 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-textfile\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.152080 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.152115 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-accelerators-collector-config\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.152134 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.152169 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.152202 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e493819d-9f49-421e-a534-9579285072be-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.152236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.152221 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c88bbb5-bae1-4930-acb2-acac3386a366-sys\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.152895 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.152877 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e493819d-9f49-421e-a534-9579285072be-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.154695 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.154671 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e493819d-9f49-421e-a534-9579285072be-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.161218 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.161180 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvvfg\" (UniqueName: \"kubernetes.io/projected/e493819d-9f49-421e-a534-9579285072be-kube-api-access-zvvfg\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.252945 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.252834 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c88bbb5-bae1-4930-acb2-acac3386a366-sys\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.252945 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.252922 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.253167 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.252947 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c88bbb5-bae1-4930-acb2-acac3386a366-sys\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.253167 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.252950 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.253167 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253096 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtpvq\" (UniqueName: \"kubernetes.io/projected/5c88bbb5-bae1-4930-acb2-acac3386a366-kube-api-access-jtpvq\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.253167 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253133 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.253167 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253159 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5c88bbb5-bae1-4930-acb2-acac3386a366-root\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.253398 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253232 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5c88bbb5-bae1-4930-acb2-acac3386a366-root\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.253398 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253263 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-wtmp\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.253398 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253315 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-tls\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.253398 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253335 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.253398 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253339 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c88bbb5-bae1-4930-acb2-acac3386a366-metrics-client-ca\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.253398 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253379 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x87j\" (UniqueName: \"kubernetes.io/projected/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-api-access-4x87j\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.253672 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253409 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-textfile\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.253672 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253432 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.253672 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:01:06.253443 2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 13 22:01:06.253672 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253462 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-wtmp\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.253672 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253467 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-accelerators-collector-config\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.253672 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:01:06.253507 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-tls podName:5c88bbb5-bae1-4930-acb2-acac3386a366 nodeName:}" failed. No retries permitted until 2026-07-13 22:01:06.753486781 +0000 UTC m=+75.936331005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-tls") pod "node-exporter-7s9sr" (UID: "5c88bbb5-bae1-4930-acb2-acac3386a366") : secret "node-exporter-tls" not found Jul 13 22:01:06.253672 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:01:06.253558 2570 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 13 22:01:06.254039 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:01:06.253680 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-tls podName:e0ff5faa-084b-44a6-84d7-bc5e25f1b41c nodeName:}" failed. No retries permitted until 2026-07-13 22:01:06.753660711 +0000 UTC m=+75.936504924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-jsfmg" (UID: "e0ff5faa-084b-44a6-84d7-bc5e25f1b41c") : secret "kube-state-metrics-tls" not found Jul 13 22:01:06.254039 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253705 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.254039 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253738 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.254039 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.253907 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c88bbb5-bae1-4930-acb2-acac3386a366-metrics-client-ca\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.254039 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.254004 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-accelerators-collector-config\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.254248 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.254111 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-textfile\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.254393 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.254349 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.254393 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.254363 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.255719 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.255696 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.256285 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.256258 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.265771 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.265750 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtpvq\" (UniqueName: \"kubernetes.io/projected/5c88bbb5-bae1-4930-acb2-acac3386a366-kube-api-access-jtpvq\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.265990 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.265967 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x87j\" (UniqueName: \"kubernetes.io/projected/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-api-access-4x87j\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.395993 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.395949 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:06.552331 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.552233 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:06.657538 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.657509 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e493819d-9f49-421e-a534-9579285072be-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.660025 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.660000 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e493819d-9f49-421e-a534-9579285072be-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mbdmm\" (UID: \"e493819d-9f49-421e-a534-9579285072be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.670127 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.670094 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:06.758496 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.758456 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-tls\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.758679 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.758517 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.761142 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.761118 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0ff5faa-084b-44a6-84d7-bc5e25f1b41c-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-jsfmg\" (UID: \"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.761263 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.761124 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5c88bbb5-bae1-4930-acb2-acac3386a366-node-exporter-tls\") pod \"node-exporter-7s9sr\" (UID: \"5c88bbb5-bae1-4930-acb2-acac3386a366\") " pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:06.781981 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.781943 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:06.924763 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.924727 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" Jul 13 22:01:06.944251 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.944224 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" Jul 13 22:01:06.970803 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:06.970776 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7s9sr" Jul 13 22:01:07.008480 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:07.008424 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:07.098098 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:07.098057 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:07.316374 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:07.316276 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:08.692454 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:08.692388 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:09.037336 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:09.037257 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:09.145904 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:09.145842 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:09.384575 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:09.384532 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:09.504178 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:09.504137 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:09.582500 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:09.582446 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:10.770569 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:10.770533 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:10.835743 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:10.835711 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:10.914295 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:10.914257 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:11.133798 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:11.133767 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:01:12.273361 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:12.273207 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:13.000828 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:01:13.000041 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c88bbb5_bae1_4930_acb2_acac3386a366.slice/crio-a548c584d655bfa1bfdb3f95fb5b836a21f0f86a81ac93a7807584ca80a9322b WatchSource:0}: Error finding container a548c584d655bfa1bfdb3f95fb5b836a21f0f86a81ac93a7807584ca80a9322b: Status 404 returned error can't find the container with id a548c584d655bfa1bfdb3f95fb5b836a21f0f86a81ac93a7807584ca80a9322b Jul 13 22:01:13.140630 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.140530 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-jsfmg"] Jul 13 22:01:13.143340 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:01:13.143317 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0ff5faa_084b_44a6_84d7_bc5e25f1b41c.slice/crio-319d6412f918648b8e53e385b51c622540694f5dadf91a6ca35af1c6c9966bc6 WatchSource:0}: Error finding container 319d6412f918648b8e53e385b51c622540694f5dadf91a6ca35af1c6c9966bc6: Status 404 returned error can't find the container with id 319d6412f918648b8e53e385b51c622540694f5dadf91a6ca35af1c6c9966bc6 Jul 13 22:01:13.156407 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.156386 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm"] Jul 13 22:01:13.159646 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:01:13.159620 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode493819d_9f49_421e_a534_9579285072be.slice/crio-7e8056e2d1101626d606d370c62a7b9d46862f5223688cbe7ffd7466909c2a6b WatchSource:0}: Error finding container 7e8056e2d1101626d606d370c62a7b9d46862f5223688cbe7ffd7466909c2a6b: Status 404 returned error can't find the container with id 7e8056e2d1101626d606d370c62a7b9d46862f5223688cbe7ffd7466909c2a6b Jul 13 22:01:13.592924 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.592804 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d77b9978f-h64zc" Jul 13 22:01:13.684523 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.684153 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-2cfgx" event={"ID":"0b9017b7-7904-4479-bde7-bf0075a083c4","Type":"ContainerStarted","Data":"a411e7f4cf1b1b4847db52a9bfc9d05915767f55464e26a3c862f6c2aa055536"} Jul 13 22:01:13.684523 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.684297 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-2cfgx" Jul 13 22:01:13.687519 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.687490 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f4c96f47-bbdtg" event={"ID":"f4cb6755-6bbf-40c8-883a-f383bdf32de2","Type":"ContainerStarted","Data":"54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74"} Jul 13 22:01:13.689219 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.689158 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7s9sr" event={"ID":"5c88bbb5-bae1-4930-acb2-acac3386a366","Type":"ContainerStarted","Data":"a548c584d655bfa1bfdb3f95fb5b836a21f0f86a81ac93a7807584ca80a9322b"} Jul 13 22:01:13.690719 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.690655 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" event={"ID":"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c","Type":"ContainerStarted","Data":"319d6412f918648b8e53e385b51c622540694f5dadf91a6ca35af1c6c9966bc6"} Jul 13 22:01:13.693773 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.693673 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" event={"ID":"bec934ce-60f6-47d0-8e29-c4ac675f1225","Type":"ContainerStarted","Data":"6b8841adab1ce65fe08d8c8f5daa1f1f88b6b059a61efc13d860ac071d0c6db7"} Jul 13 22:01:13.696479 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.696364 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" event={"ID":"e493819d-9f49-421e-a534-9579285072be","Type":"ContainerStarted","Data":"bf6ff183552fe5d121aa012166f51bc9ed194b7a50f3aa408e1d4f9bac067c9d"} Jul 13 22:01:13.696479 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.696416 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" event={"ID":"e493819d-9f49-421e-a534-9579285072be","Type":"ContainerStarted","Data":"f6e2a6624d8fb06c32fc0cbf36e5052469025b051704c2803949dc40f75ef84e"} Jul 13 22:01:13.696479 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.696452 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" event={"ID":"e493819d-9f49-421e-a534-9579285072be","Type":"ContainerStarted","Data":"7e8056e2d1101626d606d370c62a7b9d46862f5223688cbe7ffd7466909c2a6b"} Jul 13 22:01:13.696801 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.696760 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-2cfgx" Jul 13 22:01:13.706231 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.706083 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-2cfgx" podStartSLOduration=1.9282560549999999 podStartE2EDuration="22.706065485s" totalImagesPullingTime="20.77780943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:00:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:01:13.702962097 +0000 UTC m=+82.885806327" watchObservedRunningTime="2026-07-13 22:01:13.706065485 +0000 UTC m=+82.888909718" Jul 13 22:01:13.731324 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.731245 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:13.742985 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.741983 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-9sg29" podStartSLOduration=36.621506254 podStartE2EDuration="46.741911785s" totalImagesPullingTime="10.120405531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:01:13.722801158 +0000 UTC m=+82.905645390" watchObservedRunningTime="2026-07-13 22:01:13.741911785 +0000 UTC m=+82.924756016" Jul 13 22:01:13.764364 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.762766 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58f4c96f47-bbdtg" podStartSLOduration=3.6637004109999998 podStartE2EDuration="13.762745064s" totalImagesPullingTime="10.099044653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:01:13.761961699 +0000 UTC m=+82.944805941" watchObservedRunningTime="2026-07-13 22:01:13.762745064 +0000 UTC m=+82.945589295" Jul 13 22:01:13.830478 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.830432 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:13.914933 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:13.914895 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:14.015691 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:14.015650 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:14.494959 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:14.494864 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:14.513373 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:14.513297 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f4c96f47-bbdtg"] Jul 13 22:01:14.578727 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:14.578606 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:14.709233 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:14.709142 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:14.810239 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:14.810156 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:15.714244 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:15.714209 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7s9sr" event={"ID":"5c88bbb5-bae1-4930-acb2-acac3386a366","Type":"ContainerStarted","Data":"57c9ffb086ac73ab7d3ffb05abed06a0e79645f6b06787887ccd2368e0b8eee1"} Jul 13 22:01:15.716107 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:15.716024 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" event={"ID":"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c","Type":"ContainerStarted","Data":"804d99656b393710986096add766b03528f501b78c0dc65696185751ac8dd17f"} Jul 13 22:01:15.752296 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:15.752267 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:15.871018 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:15.870986 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:15.931288 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:15.931260 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:16.155009 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.154953 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" podUID="7e70b328-f83e-4dec-bb26-b5caab12b0fe" containerName="registry" containerID="cri-o://a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f" gracePeriod=30 Jul 13 22:01:16.446387 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.446353 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:01:16.551461 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.551426 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-certificates\") pod \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " Jul 13 22:01:16.551643 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.551477 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-installation-pull-secrets\") pod \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " Jul 13 22:01:16.551643 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.551515 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls\") pod \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " Jul 13 22:01:16.551643 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.551546 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76ddw\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-kube-api-access-76ddw\") pod \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " Jul 13 22:01:16.551643 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.551596 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-bound-sa-token\") pod \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " Jul 13 22:01:16.551890 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.551655 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e70b328-f83e-4dec-bb26-b5caab12b0fe-ca-trust-extracted\") pod \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " Jul 13 22:01:16.551890 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.551685 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-image-registry-private-configuration\") pod \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " Jul 13 22:01:16.551890 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.551718 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-trusted-ca\") pod \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\" (UID: \"7e70b328-f83e-4dec-bb26-b5caab12b0fe\") " Jul 13 22:01:16.552045 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.551913 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7e70b328-f83e-4dec-bb26-b5caab12b0fe" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:01:16.552312 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.552264 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7e70b328-f83e-4dec-bb26-b5caab12b0fe" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:01:16.555082 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.555048 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7e70b328-f83e-4dec-bb26-b5caab12b0fe" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:01:16.555195 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.555094 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7e70b328-f83e-4dec-bb26-b5caab12b0fe" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:01:16.555195 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.555179 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7e70b328-f83e-4dec-bb26-b5caab12b0fe" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:01:16.555312 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.555265 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7e70b328-f83e-4dec-bb26-b5caab12b0fe" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:01:16.556663 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.556641 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-kube-api-access-76ddw" (OuterVolumeSpecName: "kube-api-access-76ddw") pod "7e70b328-f83e-4dec-bb26-b5caab12b0fe" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe"). InnerVolumeSpecName "kube-api-access-76ddw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:01:16.564456 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.564424 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e70b328-f83e-4dec-bb26-b5caab12b0fe-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7e70b328-f83e-4dec-bb26-b5caab12b0fe" (UID: "7e70b328-f83e-4dec-bb26-b5caab12b0fe"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:01:16.652543 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.652509 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-bound-sa-token\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:16.652543 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.652540 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e70b328-f83e-4dec-bb26-b5caab12b0fe-ca-trust-extracted\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:16.652739 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.652553 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-image-registry-private-configuration\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:16.652739 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.652563 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-trusted-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:16.652739 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.652571 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-certificates\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:16.652739 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.652579 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e70b328-f83e-4dec-bb26-b5caab12b0fe-installation-pull-secrets\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:16.652739 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.652587 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-registry-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:16.652739 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.652595 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76ddw\" (UniqueName: \"kubernetes.io/projected/7e70b328-f83e-4dec-bb26-b5caab12b0fe-kube-api-access-76ddw\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:16.722786 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.722675 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" event={"ID":"e493819d-9f49-421e-a534-9579285072be","Type":"ContainerStarted","Data":"55a201250fdf4c5fc6f406f54d5bf30b329419246e3c66a61384ff35ca28a534"} Jul 13 22:01:16.724697 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.724667 2570 generic.go:356] "Generic (PLEG): container finished" podID="5c88bbb5-bae1-4930-acb2-acac3386a366" containerID="57c9ffb086ac73ab7d3ffb05abed06a0e79645f6b06787887ccd2368e0b8eee1" exitCode=0 Jul 13 22:01:16.724839 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.724765 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7s9sr" event={"ID":"5c88bbb5-bae1-4930-acb2-acac3386a366","Type":"ContainerDied","Data":"57c9ffb086ac73ab7d3ffb05abed06a0e79645f6b06787887ccd2368e0b8eee1"} Jul 13 22:01:16.726281 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.726256 2570 generic.go:356] "Generic (PLEG): container finished" podID="7e70b328-f83e-4dec-bb26-b5caab12b0fe" containerID="a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f" exitCode=0 Jul 13 22:01:16.726398 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.726290 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" event={"ID":"7e70b328-f83e-4dec-bb26-b5caab12b0fe","Type":"ContainerDied","Data":"a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f"} Jul 13 22:01:16.726398 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.726339 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" Jul 13 22:01:16.726398 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.726347 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85ccff9ddd-tqlj9" event={"ID":"7e70b328-f83e-4dec-bb26-b5caab12b0fe","Type":"ContainerDied","Data":"a3b9e57389416c7049cdffe995a9723d3225e1201f640c5a6700b139f6b3b6e4"} Jul 13 22:01:16.726398 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.726380 2570 scope.go:117] "RemoveContainer" containerID="a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f" Jul 13 22:01:16.729479 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.729456 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" event={"ID":"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c","Type":"ContainerStarted","Data":"cf70583a9987a2d0aa2f32f882bee3f9eeca29e67ba09cda5525f8c8faa65c3e"} Jul 13 22:01:16.729585 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.729507 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" event={"ID":"e0ff5faa-084b-44a6-84d7-bc5e25f1b41c","Type":"ContainerStarted","Data":"d881e051ea7e8b4a821bfa6bd8fe0704fe8e3abc64f08d099e0f5dc34ea8a69b"} Jul 13 22:01:16.737047 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.737028 2570 scope.go:117] "RemoveContainer" containerID="a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f" Jul 13 22:01:16.737353 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:01:16.737328 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f\": container with ID starting with a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f not found: ID does not exist" containerID="a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f" Jul 13 22:01:16.737436 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.737363 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f"} err="failed to get container status \"a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f\": rpc error: code = NotFound desc = could not find container \"a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f\": container with ID starting with a59d9608164c551c4a161ec2043f7b9cb33eaf47898f201599f7828845747e0f not found: ID does not exist" Jul 13 22:01:16.744116 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.743524 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mbdmm" podStartSLOduration=9.170893124 podStartE2EDuration="11.743506251s" totalImagesPullingTime="2.572613127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:01:16.742804867 +0000 UTC m=+85.925649117" watchObservedRunningTime="2026-07-13 22:01:16.743506251 +0000 UTC m=+85.926350483" Jul 13 22:01:16.760641 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.760615 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85ccff9ddd-tqlj9"] Jul 13 22:01:16.768431 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.768404 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-85ccff9ddd-tqlj9"] Jul 13 22:01:16.789920 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:16.789868 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-jsfmg" podStartSLOduration=8.367751305 podStartE2EDuration="10.789830899s" totalImagesPullingTime="2.422079594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:01:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:01:16.787636608 +0000 UTC m=+85.970480864" watchObservedRunningTime="2026-07-13 22:01:16.789830899 +0000 UTC m=+85.972675132" Jul 13 22:01:17.317874 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:17.317775 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e70b328-f83e-4dec-bb26-b5caab12b0fe" path="/var/lib/kubelet/pods/7e70b328-f83e-4dec-bb26-b5caab12b0fe/volumes" Jul 13 22:01:17.737138 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:17.737094 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7s9sr" event={"ID":"5c88bbb5-bae1-4930-acb2-acac3386a366","Type":"ContainerStarted","Data":"9c1d092882d84e92b8f0aca13bb297930ced9c1e7127c5a0d5d37f621351c4bf"} Jul 13 22:01:17.737587 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:17.737161 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7s9sr" event={"ID":"5c88bbb5-bae1-4930-acb2-acac3386a366","Type":"ContainerStarted","Data":"fe34f9574e24e2cd03cf9067a0df633db54293f026109781db2e49056f045d7b"} Jul 13 22:01:17.762127 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:17.762067 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7s9sr" podStartSLOduration=9.0936227 podStartE2EDuration="11.762050542s" totalImagesPullingTime="2.55974265s" totalInitContainerRuntime="108.685192ms" isStatefulPod=true podCreationTimestamp="2026-07-13 22:01:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:01:17.760784346 +0000 UTC m=+86.943628603" watchObservedRunningTime="2026-07-13 22:01:17.762050542 +0000 UTC m=+86.944894773" Jul 13 22:01:19.448822 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:19.448779 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:19.542394 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:19.542359 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:20.394811 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:20.394773 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:21.056573 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:21.056520 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:21.125474 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:21.125437 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:22.940333 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:22.940301 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:23.033147 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:23.033115 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:24.155908 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:24.155869 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:24.262288 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:24.262253 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:25.758384 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:25.758350 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:25.858797 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:25.858743 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:25.958810 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:25.958779 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:28.508834 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:28.508791 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:28.606104 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:28.606069 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:34.650385 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:34.650354 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rnlqj" Jul 13 22:01:37.903469 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:37.903433 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:37.959075 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:37.959043 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:40.738966 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:40.738927 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58f4c96f47-bbdtg" podUID="f4cb6755-6bbf-40c8-883a-f383bdf32de2" containerName="console" containerID="cri-o://54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74" gracePeriod=15 Jul 13 22:01:41.010864 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.010794 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:41.047541 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.047516 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-oauth-config\") pod \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " Jul 13 22:01:41.047698 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.047559 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-config\") pod \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " Jul 13 22:01:41.047698 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.047585 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5n4p\" (UniqueName: \"kubernetes.io/projected/f4cb6755-6bbf-40c8-883a-f383bdf32de2-kube-api-access-v5n4p\") pod \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " Jul 13 22:01:41.047698 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.047605 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-oauth-serving-cert\") pod \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " Jul 13 22:01:41.047698 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.047623 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-service-ca\") pod \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " Jul 13 22:01:41.047698 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.047653 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-serving-cert\") pod \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " Jul 13 22:01:41.047698 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.047687 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-trusted-ca-bundle\") pod \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\" (UID: \"f4cb6755-6bbf-40c8-883a-f383bdf32de2\") " Jul 13 22:01:41.048116 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.048074 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-config" (OuterVolumeSpecName: "console-config") pod "f4cb6755-6bbf-40c8-883a-f383bdf32de2" (UID: "f4cb6755-6bbf-40c8-883a-f383bdf32de2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:01:41.048197 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.048114 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f4cb6755-6bbf-40c8-883a-f383bdf32de2" (UID: "f4cb6755-6bbf-40c8-883a-f383bdf32de2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:01:41.048197 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.048135 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f4cb6755-6bbf-40c8-883a-f383bdf32de2" (UID: "f4cb6755-6bbf-40c8-883a-f383bdf32de2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:01:41.048408 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.048384 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-service-ca" (OuterVolumeSpecName: "service-ca") pod "f4cb6755-6bbf-40c8-883a-f383bdf32de2" (UID: "f4cb6755-6bbf-40c8-883a-f383bdf32de2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:01:41.050058 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.050028 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f4cb6755-6bbf-40c8-883a-f383bdf32de2" (UID: "f4cb6755-6bbf-40c8-883a-f383bdf32de2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:01:41.050161 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.050081 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4cb6755-6bbf-40c8-883a-f383bdf32de2-kube-api-access-v5n4p" (OuterVolumeSpecName: "kube-api-access-v5n4p") pod "f4cb6755-6bbf-40c8-883a-f383bdf32de2" (UID: "f4cb6755-6bbf-40c8-883a-f383bdf32de2"). InnerVolumeSpecName "kube-api-access-v5n4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:01:41.050224 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.050197 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f4cb6755-6bbf-40c8-883a-f383bdf32de2" (UID: "f4cb6755-6bbf-40c8-883a-f383bdf32de2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:01:41.148788 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.148764 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:41.148788 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.148784 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-serving-cert\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:41.148947 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.148794 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-trusted-ca-bundle\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:41.148947 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.148803 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-oauth-config\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:41.148947 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.148811 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-console-config\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:41.148947 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.148819 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5n4p\" (UniqueName: \"kubernetes.io/projected/f4cb6755-6bbf-40c8-883a-f383bdf32de2-kube-api-access-v5n4p\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:41.148947 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.148828 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4cb6755-6bbf-40c8-883a-f383bdf32de2-oauth-serving-cert\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:01:41.825725 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.825686 2570 generic.go:356] "Generic (PLEG): container finished" podID="f4cb6755-6bbf-40c8-883a-f383bdf32de2" containerID="54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74" exitCode=0 Jul 13 22:01:41.826166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.825744 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f4c96f47-bbdtg" Jul 13 22:01:41.826166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.825770 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f4c96f47-bbdtg" event={"ID":"f4cb6755-6bbf-40c8-883a-f383bdf32de2","Type":"ContainerDied","Data":"54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74"} Jul 13 22:01:41.826166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.825827 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f4c96f47-bbdtg" event={"ID":"f4cb6755-6bbf-40c8-883a-f383bdf32de2","Type":"ContainerDied","Data":"17be1af0dc4310ebe6e7a9ca08e56bfac7bff522cd393d99e65bae9048cdd65a"} Jul 13 22:01:41.826166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.825867 2570 scope.go:117] "RemoveContainer" containerID="54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74" Jul 13 22:01:41.833778 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.833759 2570 scope.go:117] "RemoveContainer" containerID="54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74" Jul 13 22:01:41.834043 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:01:41.834024 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74\": container with ID starting with 54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74 not found: ID does not exist" containerID="54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74" Jul 13 22:01:41.834096 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.834052 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74"} err="failed to get container status \"54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74\": rpc error: code = NotFound desc = could not find container \"54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74\": container with ID starting with 54c6e6b7fd054e71470339477d8e3ed54dd3716a5b951c0bde4495e957938b74 not found: ID does not exist" Jul 13 22:01:41.839504 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.839482 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f4c96f47-bbdtg"] Jul 13 22:01:41.843611 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:41.843590 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58f4c96f47-bbdtg"] Jul 13 22:01:43.316123 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:43.316093 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4cb6755-6bbf-40c8-883a-f383bdf32de2" path="/var/lib/kubelet/pods/f4cb6755-6bbf-40c8-883a-f383bdf32de2/volumes" Jul 13 22:01:48.543481 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:01:48.543444 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:25.362377 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:25.362345 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:25.466792 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:25.466755 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:30.464098 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.464069 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb"] Jul 13 22:02:30.464463 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.464358 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4cb6755-6bbf-40c8-883a-f383bdf32de2" containerName="console" Jul 13 22:02:30.464463 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.464371 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4cb6755-6bbf-40c8-883a-f383bdf32de2" containerName="console" Jul 13 22:02:30.464463 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.464381 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e70b328-f83e-4dec-bb26-b5caab12b0fe" containerName="registry" Jul 13 22:02:30.464463 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.464387 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e70b328-f83e-4dec-bb26-b5caab12b0fe" containerName="registry" Jul 13 22:02:30.464463 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.464436 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4cb6755-6bbf-40c8-883a-f383bdf32de2" containerName="console" Jul 13 22:02:30.464463 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.464442 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e70b328-f83e-4dec-bb26-b5caab12b0fe" containerName="registry" Jul 13 22:02:30.466842 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.466818 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.469145 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.469117 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 22:02:30.469270 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.469129 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 22:02:30.469270 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.469159 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 22:02:30.469430 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.469414 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 22:02:30.469942 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.469918 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8w9h6\"" Jul 13 22:02:30.472251 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.470115 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 22:02:30.477202 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.477179 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 22:02:30.485243 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.485221 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb"] Jul 13 22:02:30.590071 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.590024 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-telemeter-client-tls\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.590071 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.590068 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eab8f0c-317c-4a8d-9682-3084d30974d9-metrics-client-ca\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.590071 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.590085 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-federate-client-tls\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.590376 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.590169 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-secret-telemeter-client\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.590376 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.590210 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.590376 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.590229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eab8f0c-317c-4a8d-9682-3084d30974d9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.590376 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.590271 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eab8f0c-317c-4a8d-9682-3084d30974d9-serving-certs-ca-bundle\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.590376 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.590319 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8cb7\" (UniqueName: \"kubernetes.io/projected/3eab8f0c-317c-4a8d-9682-3084d30974d9-kube-api-access-w8cb7\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.691210 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.691125 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-secret-telemeter-client\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.691210 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.691172 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.691210 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.691192 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eab8f0c-317c-4a8d-9682-3084d30974d9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.691427 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.691232 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eab8f0c-317c-4a8d-9682-3084d30974d9-serving-certs-ca-bundle\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.691427 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.691254 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8cb7\" (UniqueName: \"kubernetes.io/projected/3eab8f0c-317c-4a8d-9682-3084d30974d9-kube-api-access-w8cb7\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.691427 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.691279 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-telemeter-client-tls\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.691427 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.691415 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eab8f0c-317c-4a8d-9682-3084d30974d9-metrics-client-ca\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.691704 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.691676 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-federate-client-tls\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.692111 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.692085 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eab8f0c-317c-4a8d-9682-3084d30974d9-serving-certs-ca-bundle\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.692287 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.692248 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eab8f0c-317c-4a8d-9682-3084d30974d9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.692393 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.692219 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eab8f0c-317c-4a8d-9682-3084d30974d9-metrics-client-ca\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.693838 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.693814 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-secret-telemeter-client\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.693957 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.693837 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.694013 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.693958 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-federate-client-tls\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.694088 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.694068 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3eab8f0c-317c-4a8d-9682-3084d30974d9-telemeter-client-tls\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.701047 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.701030 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8cb7\" (UniqueName: \"kubernetes.io/projected/3eab8f0c-317c-4a8d-9682-3084d30974d9-kube-api-access-w8cb7\") pod \"telemeter-client-5c77fb7c4-k7wcb\" (UID: \"3eab8f0c-317c-4a8d-9682-3084d30974d9\") " pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.778475 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.778449 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" Jul 13 22:02:30.908032 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.908009 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb"] Jul 13 22:02:30.910911 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:02:30.910886 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eab8f0c_317c_4a8d_9682_3084d30974d9.slice/crio-c76e480a0e9fd8b2650127bb286235109c6618cddb676c6cf505aca17e906292 WatchSource:0}: Error finding container c76e480a0e9fd8b2650127bb286235109c6618cddb676c6cf505aca17e906292: Status 404 returned error can't find the container with id c76e480a0e9fd8b2650127bb286235109c6618cddb676c6cf505aca17e906292 Jul 13 22:02:30.993795 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:30.993723 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" event={"ID":"3eab8f0c-317c-4a8d-9682-3084d30974d9","Type":"ContainerStarted","Data":"c76e480a0e9fd8b2650127bb286235109c6618cddb676c6cf505aca17e906292"} Jul 13 22:02:34.005943 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:34.005899 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" event={"ID":"3eab8f0c-317c-4a8d-9682-3084d30974d9","Type":"ContainerStarted","Data":"f00ad64b8be6304b5c727bb2e2b57945f784d7305b1ca0d1961f6a3206e4eb79"} Jul 13 22:02:35.011578 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:35.011544 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" event={"ID":"3eab8f0c-317c-4a8d-9682-3084d30974d9","Type":"ContainerStarted","Data":"1e34e982a1eaa6b08b1883010d87154c4deff612fc90482d394b0b5177405e12"} Jul 13 22:02:35.011962 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:35.011599 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" event={"ID":"3eab8f0c-317c-4a8d-9682-3084d30974d9","Type":"ContainerStarted","Data":"7cd014b509b6de885137a6e5877d00f9e6da448ad0b2fb1c5aedf1bd7bbbea01"} Jul 13 22:02:35.031470 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:35.031416 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5c77fb7c4-k7wcb" podStartSLOduration=1.513843675 podStartE2EDuration="5.031401745s" totalImagesPullingTime="3.51755807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:02:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:02:35.030831533 +0000 UTC m=+164.213675764" watchObservedRunningTime="2026-07-13 22:02:35.031401745 +0000 UTC m=+164.214245975" Jul 13 22:02:36.597203 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.597172 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8b9c6d98f-bkzjf"] Jul 13 22:02:36.599102 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.599085 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.601240 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.601220 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 22:02:36.601715 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.601691 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 22:02:36.602020 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.601736 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-crbrs\"" Jul 13 22:02:36.602020 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.601782 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 22:02:36.602020 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.601798 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 22:02:36.602020 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.601801 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 22:02:36.606423 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.606399 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 22:02:36.612210 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.612164 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8b9c6d98f-bkzjf"] Jul 13 22:02:36.649927 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.649901 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:36.707433 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.707414 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:36.738403 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.738385 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-oauth-config\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.738485 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.738412 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-trusted-ca-bundle\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.738485 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.738438 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt754\" (UniqueName: \"kubernetes.io/projected/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-kube-api-access-rt754\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.738485 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.738478 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-service-ca\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.738587 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.738515 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-serving-cert\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.738587 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.738546 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-oauth-serving-cert\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.738587 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.738569 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-config\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.839658 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.839630 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-oauth-config\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.839658 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.839667 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-trusted-ca-bundle\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.839786 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.839694 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt754\" (UniqueName: \"kubernetes.io/projected/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-kube-api-access-rt754\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.839903 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.839882 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-service-ca\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.839959 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.839934 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-serving-cert\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.840002 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.839978 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-oauth-serving-cert\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.840041 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.840014 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-config\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.840630 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.840568 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-service-ca\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.840720 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.840707 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-oauth-serving-cert\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.840769 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.840721 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-config\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.840769 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.840708 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-trusted-ca-bundle\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.842078 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.842047 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-oauth-config\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.842299 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.842280 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-serving-cert\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.853711 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.853662 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt754\" (UniqueName: \"kubernetes.io/projected/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-kube-api-access-rt754\") pod \"console-8b9c6d98f-bkzjf\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:36.908698 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:36.908672 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:37.051124 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:37.048234 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8b9c6d98f-bkzjf"] Jul 13 22:02:38.023372 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:38.023337 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b9c6d98f-bkzjf" event={"ID":"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a","Type":"ContainerStarted","Data":"6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295"} Jul 13 22:02:38.023738 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:38.023394 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b9c6d98f-bkzjf" event={"ID":"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a","Type":"ContainerStarted","Data":"28a71fb55d0fc35d610e8537bf48b815bfe9a5d8f15990689feaf395936be1ff"} Jul 13 22:02:38.041122 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:38.041074 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8b9c6d98f-bkzjf" podStartSLOduration=2.041047216 podStartE2EDuration="2.041047216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:02:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:02:38.040496165 +0000 UTC m=+167.223340407" watchObservedRunningTime="2026-07-13 22:02:38.041047216 +0000 UTC m=+167.223891447" Jul 13 22:02:40.639281 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:40.639247 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:40.709155 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:40.709126 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:46.909874 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:46.909775 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:46.910247 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:46.909879 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:46.914515 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:46.914495 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:47.059837 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.059811 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:02:47.209721 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.209645 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8ff766d9d-gdrhl"] Jul 13 22:02:47.211581 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.211565 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.232869 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.232827 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8ff766d9d-gdrhl"] Jul 13 22:02:47.319586 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.319558 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-oauth-config\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.319586 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.319586 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-oauth-serving-cert\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.319802 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.319612 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-serving-cert\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.319802 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.319630 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-trusted-ca-bundle\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.319802 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.319708 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-service-ca\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.319802 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.319747 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28hfr\" (UniqueName: \"kubernetes.io/projected/9494c095-f074-45f3-b8a2-5154f589add4-kube-api-access-28hfr\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.319802 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.319773 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-console-config\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.420490 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.420457 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28hfr\" (UniqueName: \"kubernetes.io/projected/9494c095-f074-45f3-b8a2-5154f589add4-kube-api-access-28hfr\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.420625 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.420500 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-console-config\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.420625 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.420532 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-oauth-config\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.420625 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.420553 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-oauth-serving-cert\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.420625 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.420594 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-serving-cert\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.420625 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.420620 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-trusted-ca-bundle\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.420880 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.420665 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-service-ca\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.421358 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.421336 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-oauth-serving-cert\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.421451 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.421398 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-console-config\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.421491 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.421460 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-service-ca\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.421537 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.421521 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-trusted-ca-bundle\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.423132 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.423107 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-serving-cert\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.423247 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.423225 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-oauth-config\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.428376 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.428355 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28hfr\" (UniqueName: \"kubernetes.io/projected/9494c095-f074-45f3-b8a2-5154f589add4-kube-api-access-28hfr\") pod \"console-8ff766d9d-gdrhl\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.520482 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.520405 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:47.642926 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:47.642891 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8ff766d9d-gdrhl"] Jul 13 22:02:47.645427 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:02:47.645400 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9494c095_f074_45f3_b8a2_5154f589add4.slice/crio-56a3ccc988e79b719582dd9d88c1424bc59228ed26ee8fed0d8c5fc5f9f4ea21 WatchSource:0}: Error finding container 56a3ccc988e79b719582dd9d88c1424bc59228ed26ee8fed0d8c5fc5f9f4ea21: Status 404 returned error can't find the container with id 56a3ccc988e79b719582dd9d88c1424bc59228ed26ee8fed0d8c5fc5f9f4ea21 Jul 13 22:02:48.061461 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:48.061429 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8ff766d9d-gdrhl" event={"ID":"9494c095-f074-45f3-b8a2-5154f589add4","Type":"ContainerStarted","Data":"185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5"} Jul 13 22:02:48.061893 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:48.061477 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8ff766d9d-gdrhl" event={"ID":"9494c095-f074-45f3-b8a2-5154f589add4","Type":"ContainerStarted","Data":"56a3ccc988e79b719582dd9d88c1424bc59228ed26ee8fed0d8c5fc5f9f4ea21"} Jul 13 22:02:48.079091 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:48.079044 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8ff766d9d-gdrhl" podStartSLOduration=1.079027737 podStartE2EDuration="1.079027737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:02:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:02:48.07809715 +0000 UTC m=+177.260941381" watchObservedRunningTime="2026-07-13 22:02:48.079027737 +0000 UTC m=+177.261871968" Jul 13 22:02:50.526155 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:50.526113 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:52.599570 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:52.599538 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:52.666484 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:52.666447 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:57.520693 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:57.520647 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:57.521157 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:57.520710 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:57.525044 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:57.525025 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:58.101950 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:58.101922 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:02:58.144016 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:58.143982 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8b9c6d98f-bkzjf"] Jul 13 22:02:58.276356 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:58.276319 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:58.378042 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:58.378009 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:02:58.459203 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:02:58.459180 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:07.952951 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:07.952918 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:08.047532 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:08.047492 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:18.640409 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:18.640378 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:18.720690 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:18.720649 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:23.164623 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.164576 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8b9c6d98f-bkzjf" podUID="1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" containerName="console" containerID="cri-o://6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295" gracePeriod=15 Jul 13 22:03:23.408314 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.408288 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:03:23.502802 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.502719 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-config\") pod \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " Jul 13 22:03:23.502802 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.502795 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt754\" (UniqueName: \"kubernetes.io/projected/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-kube-api-access-rt754\") pod \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " Jul 13 22:03:23.503037 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.502815 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-service-ca\") pod \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " Jul 13 22:03:23.503037 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.502833 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-serving-cert\") pod \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " Jul 13 22:03:23.503037 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.502883 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-trusted-ca-bundle\") pod \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " Jul 13 22:03:23.503037 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.502910 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-oauth-config\") pod \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " Jul 13 22:03:23.503037 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.502935 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-oauth-serving-cert\") pod \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\" (UID: \"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a\") " Jul 13 22:03:23.503285 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.503263 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-config" (OuterVolumeSpecName: "console-config") pod "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" (UID: "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:03:23.503366 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.503341 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-service-ca" (OuterVolumeSpecName: "service-ca") pod "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" (UID: "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:03:23.503424 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.503369 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" (UID: "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:03:23.503424 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.503401 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" (UID: "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:03:23.504992 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.504961 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" (UID: "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:03:23.505088 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.505002 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" (UID: "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:03:23.505088 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.505024 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-kube-api-access-rt754" (OuterVolumeSpecName: "kube-api-access-rt754") pod "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" (UID: "1f8e834b-cc5d-46d9-81f9-9d9e3fae739a"). InnerVolumeSpecName "kube-api-access-rt754". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:03:23.603726 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.603698 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt754\" (UniqueName: \"kubernetes.io/projected/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-kube-api-access-rt754\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:03:23.603726 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.603723 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:03:23.603915 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.603736 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-serving-cert\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:03:23.603915 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.603749 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-trusted-ca-bundle\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:03:23.603915 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.603771 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-oauth-config\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:03:23.603915 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.603783 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-oauth-serving-cert\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:03:23.603915 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:23.603795 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a-console-config\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:03:24.194624 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:24.194585 2570 generic.go:356] "Generic (PLEG): container finished" podID="1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" containerID="6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295" exitCode=0 Jul 13 22:03:24.195078 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:24.194649 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b9c6d98f-bkzjf" Jul 13 22:03:24.195078 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:24.194652 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b9c6d98f-bkzjf" event={"ID":"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a","Type":"ContainerDied","Data":"6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295"} Jul 13 22:03:24.195078 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:24.194789 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b9c6d98f-bkzjf" event={"ID":"1f8e834b-cc5d-46d9-81f9-9d9e3fae739a","Type":"ContainerDied","Data":"28a71fb55d0fc35d610e8537bf48b815bfe9a5d8f15990689feaf395936be1ff"} Jul 13 22:03:24.195078 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:24.194819 2570 scope.go:117] "RemoveContainer" containerID="6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295" Jul 13 22:03:24.202959 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:24.202940 2570 scope.go:117] "RemoveContainer" containerID="6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295" Jul 13 22:03:24.203209 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:03:24.203192 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295\": container with ID starting with 6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295 not found: ID does not exist" containerID="6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295" Jul 13 22:03:24.203251 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:24.203218 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295"} err="failed to get container status \"6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295\": rpc error: code = NotFound desc = could not find container \"6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295\": container with ID starting with 6a6c0076f735680733edb55765edcd24f0a9a768810f3b7f56283e714243d295 not found: ID does not exist" Jul 13 22:03:24.215294 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:24.215264 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8b9c6d98f-bkzjf"] Jul 13 22:03:24.219249 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:24.219229 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8b9c6d98f-bkzjf"] Jul 13 22:03:25.315097 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:25.315064 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" path="/var/lib/kubelet/pods/1f8e834b-cc5d-46d9-81f9-9d9e3fae739a/volumes" Jul 13 22:03:26.300762 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:26.300722 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:26.403522 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:26.403497 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:53.950818 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:53.950783 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:55.316227 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.316194 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:55.846340 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.846311 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8"] Jul 13 22:03:55.846592 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.846581 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" containerName="console" Jul 13 22:03:55.846636 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.846594 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" containerName="console" Jul 13 22:03:55.846669 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.846644 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f8e834b-cc5d-46d9-81f9-9d9e3fae739a" containerName="console" Jul 13 22:03:55.850693 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.850676 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8" Jul 13 22:03:55.852610 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.852583 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 22:03:55.852720 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.852619 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 22:03:55.852720 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.852654 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-5fhgz\"" Jul 13 22:03:55.864227 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.864207 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8"] Jul 13 22:03:55.941385 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.941359 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97515edb-e70a-44c3-a960-a37422f5567e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zhlj8\" (UID: \"97515edb-e70a-44c3-a960-a37422f5567e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8" Jul 13 22:03:55.941501 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:55.941407 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm9f9\" (UniqueName: \"kubernetes.io/projected/97515edb-e70a-44c3-a960-a37422f5567e-kube-api-access-tm9f9\") pod \"cert-manager-operator-controller-manager-55c444bf86-zhlj8\" (UID: \"97515edb-e70a-44c3-a960-a37422f5567e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8" Jul 13 22:03:56.042107 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:56.042085 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97515edb-e70a-44c3-a960-a37422f5567e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zhlj8\" (UID: \"97515edb-e70a-44c3-a960-a37422f5567e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8" Jul 13 22:03:56.042196 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:56.042131 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm9f9\" (UniqueName: \"kubernetes.io/projected/97515edb-e70a-44c3-a960-a37422f5567e-kube-api-access-tm9f9\") pod \"cert-manager-operator-controller-manager-55c444bf86-zhlj8\" (UID: \"97515edb-e70a-44c3-a960-a37422f5567e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8" Jul 13 22:03:56.042427 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:56.042410 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97515edb-e70a-44c3-a960-a37422f5567e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zhlj8\" (UID: \"97515edb-e70a-44c3-a960-a37422f5567e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8" Jul 13 22:03:56.049859 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:56.049837 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm9f9\" (UniqueName: \"kubernetes.io/projected/97515edb-e70a-44c3-a960-a37422f5567e-kube-api-access-tm9f9\") pod \"cert-manager-operator-controller-manager-55c444bf86-zhlj8\" (UID: \"97515edb-e70a-44c3-a960-a37422f5567e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8" Jul 13 22:03:56.159275 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:56.159249 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8" Jul 13 22:03:56.288037 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:56.288007 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8"] Jul 13 22:03:56.289338 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:03:56.289315 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97515edb_e70a_44c3_a960_a37422f5567e.slice/crio-e31783fd5ce6af20d31ab1944d004f96d7f69bc9bedbf0818b785d6c8e304552 WatchSource:0}: Error finding container e31783fd5ce6af20d31ab1944d004f96d7f69bc9bedbf0818b785d6c8e304552: Status 404 returned error can't find the container with id e31783fd5ce6af20d31ab1944d004f96d7f69bc9bedbf0818b785d6c8e304552 Jul 13 22:03:56.313577 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:56.313549 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8" event={"ID":"97515edb-e70a-44c3-a960-a37422f5567e","Type":"ContainerStarted","Data":"e31783fd5ce6af20d31ab1944d004f96d7f69bc9bedbf0818b785d6c8e304552"} Jul 13 22:03:57.552161 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:57.552122 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:57.655879 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:57.655820 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:59.328665 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:59.328592 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8" event={"ID":"97515edb-e70a-44c3-a960-a37422f5567e","Type":"ContainerStarted","Data":"85b6f4a84847877a66359c761aa1a9592466b2ea3f74434bbb56f9d76cbdd540"} Jul 13 22:03:59.350198 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:59.350146 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zhlj8" podStartSLOduration=1.7247500310000001 podStartE2EDuration="4.350125911s" totalImagesPullingTime="2.62537588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:03:59.349919649 +0000 UTC m=+248.532763901" watchObservedRunningTime="2026-07-13 22:03:59.350125911 +0000 UTC m=+248.532970142" Jul 13 22:03:59.602200 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:59.602122 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:59.709446 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:59.709415 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:59.814490 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:59.814457 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:59.969113 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:59.969084 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zdcfk"] Jul 13 22:03:59.972495 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:59.972479 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" Jul 13 22:03:59.974416 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:59.974398 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-r7562\"" Jul 13 22:03:59.974907 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:59.974891 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 22:03:59.974962 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:59.974934 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 22:03:59.984999 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:03:59.984978 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zdcfk"] Jul 13 22:04:00.079136 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:00.079101 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88b119fd-19f4-4648-bccb-c09d0f4440b8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zdcfk\" (UID: \"88b119fd-19f4-4648-bccb-c09d0f4440b8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" Jul 13 22:04:00.079289 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:00.079144 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk7qj\" (UniqueName: \"kubernetes.io/projected/88b119fd-19f4-4648-bccb-c09d0f4440b8-kube-api-access-jk7qj\") pod \"cert-manager-webhook-6486ffd997-zdcfk\" (UID: \"88b119fd-19f4-4648-bccb-c09d0f4440b8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" Jul 13 22:04:00.180150 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:00.180116 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88b119fd-19f4-4648-bccb-c09d0f4440b8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zdcfk\" (UID: \"88b119fd-19f4-4648-bccb-c09d0f4440b8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" Jul 13 22:04:00.180294 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:00.180156 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk7qj\" (UniqueName: \"kubernetes.io/projected/88b119fd-19f4-4648-bccb-c09d0f4440b8-kube-api-access-jk7qj\") pod \"cert-manager-webhook-6486ffd997-zdcfk\" (UID: \"88b119fd-19f4-4648-bccb-c09d0f4440b8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" Jul 13 22:04:00.194264 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:00.194234 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk7qj\" (UniqueName: \"kubernetes.io/projected/88b119fd-19f4-4648-bccb-c09d0f4440b8-kube-api-access-jk7qj\") pod \"cert-manager-webhook-6486ffd997-zdcfk\" (UID: \"88b119fd-19f4-4648-bccb-c09d0f4440b8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" Jul 13 22:04:00.194390 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:00.194374 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88b119fd-19f4-4648-bccb-c09d0f4440b8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zdcfk\" (UID: \"88b119fd-19f4-4648-bccb-c09d0f4440b8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" Jul 13 22:04:00.297267 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:00.297193 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" Jul 13 22:04:00.425592 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:00.425569 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zdcfk"] Jul 13 22:04:00.427723 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:04:00.427701 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88b119fd_19f4_4648_bccb_c09d0f4440b8.slice/crio-f7640fa0704b4e03fcd97db63e4f1faa0a66dbb1fc77295d8f07ba2fe99e2c44 WatchSource:0}: Error finding container f7640fa0704b4e03fcd97db63e4f1faa0a66dbb1fc77295d8f07ba2fe99e2c44: Status 404 returned error can't find the container with id f7640fa0704b4e03fcd97db63e4f1faa0a66dbb1fc77295d8f07ba2fe99e2c44 Jul 13 22:04:01.325779 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:01.325732 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:01.337425 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:01.337400 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" event={"ID":"88b119fd-19f4-4648-bccb-c09d0f4440b8","Type":"ContainerStarted","Data":"f7640fa0704b4e03fcd97db63e4f1faa0a66dbb1fc77295d8f07ba2fe99e2c44"} Jul 13 22:04:01.956933 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:01.956898 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:02.223823 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:02.223747 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:04.747478 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:04.747437 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:04.814261 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:04.814193 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:05.354146 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:05.354060 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" event={"ID":"88b119fd-19f4-4648-bccb-c09d0f4440b8","Type":"ContainerStarted","Data":"00a1cf3875cfe800d2343692adaeb0e7741569c908d5fa3e2c3795821a714cc7"} Jul 13 22:04:05.354146 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:05.354115 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" Jul 13 22:04:05.379273 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:05.379221 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" podStartSLOduration=1.79632252 podStartE2EDuration="6.379205318s" totalImagesPullingTime="4.582882798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:04:05.377902296 +0000 UTC m=+254.560746527" watchObservedRunningTime="2026-07-13 22:04:05.379205318 +0000 UTC m=+254.562049548" Jul 13 22:04:09.572591 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:09.572543 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:10.868450 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:10.868419 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:10.980494 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:10.980454 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:11.359814 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:11.359791 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zdcfk" Jul 13 22:04:11.937503 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:11.937470 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nx96n"] Jul 13 22:04:11.940543 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:11.940524 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nx96n" Jul 13 22:04:11.942459 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:11.942443 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-n5n27\"" Jul 13 22:04:11.948424 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:11.948401 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nx96n"] Jul 13 22:04:11.975553 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:11.975529 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edf1d451-84eb-4bd4-9433-a983efffc8b0-bound-sa-token\") pod \"cert-manager-55b4d954d5-nx96n\" (UID: \"edf1d451-84eb-4bd4-9433-a983efffc8b0\") " pod="cert-manager/cert-manager-55b4d954d5-nx96n" Jul 13 22:04:11.975656 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:11.975565 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44r5f\" (UniqueName: \"kubernetes.io/projected/edf1d451-84eb-4bd4-9433-a983efffc8b0-kube-api-access-44r5f\") pod \"cert-manager-55b4d954d5-nx96n\" (UID: \"edf1d451-84eb-4bd4-9433-a983efffc8b0\") " pod="cert-manager/cert-manager-55b4d954d5-nx96n" Jul 13 22:04:12.076793 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:12.076766 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edf1d451-84eb-4bd4-9433-a983efffc8b0-bound-sa-token\") pod \"cert-manager-55b4d954d5-nx96n\" (UID: \"edf1d451-84eb-4bd4-9433-a983efffc8b0\") " pod="cert-manager/cert-manager-55b4d954d5-nx96n" Jul 13 22:04:12.076932 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:12.076813 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44r5f\" (UniqueName: \"kubernetes.io/projected/edf1d451-84eb-4bd4-9433-a983efffc8b0-kube-api-access-44r5f\") pod \"cert-manager-55b4d954d5-nx96n\" (UID: \"edf1d451-84eb-4bd4-9433-a983efffc8b0\") " pod="cert-manager/cert-manager-55b4d954d5-nx96n" Jul 13 22:04:12.084236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:12.084217 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edf1d451-84eb-4bd4-9433-a983efffc8b0-bound-sa-token\") pod \"cert-manager-55b4d954d5-nx96n\" (UID: \"edf1d451-84eb-4bd4-9433-a983efffc8b0\") " pod="cert-manager/cert-manager-55b4d954d5-nx96n" Jul 13 22:04:12.084424 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:12.084406 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44r5f\" (UniqueName: \"kubernetes.io/projected/edf1d451-84eb-4bd4-9433-a983efffc8b0-kube-api-access-44r5f\") pod \"cert-manager-55b4d954d5-nx96n\" (UID: \"edf1d451-84eb-4bd4-9433-a983efffc8b0\") " pod="cert-manager/cert-manager-55b4d954d5-nx96n" Jul 13 22:04:12.249266 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:12.249187 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nx96n" Jul 13 22:04:12.370155 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:12.370129 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nx96n"] Jul 13 22:04:12.372319 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:04:12.372283 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podedf1d451_84eb_4bd4_9433_a983efffc8b0.slice/crio-e87ab371796b4560457824bbf38cf2abe7c63aa47e9ecdfc70f045a37093d08c WatchSource:0}: Error finding container e87ab371796b4560457824bbf38cf2abe7c63aa47e9ecdfc70f045a37093d08c: Status 404 returned error can't find the container with id e87ab371796b4560457824bbf38cf2abe7c63aa47e9ecdfc70f045a37093d08c Jul 13 22:04:12.381426 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:12.381396 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nx96n" event={"ID":"edf1d451-84eb-4bd4-9433-a983efffc8b0","Type":"ContainerStarted","Data":"e87ab371796b4560457824bbf38cf2abe7c63aa47e9ecdfc70f045a37093d08c"} Jul 13 22:04:13.387583 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:13.387506 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nx96n" event={"ID":"edf1d451-84eb-4bd4-9433-a983efffc8b0","Type":"ContainerStarted","Data":"5df695c179dfe946cdf124c58a2101bdc57bfea063695d09281f195a5496c77a"} Jul 13 22:04:13.404915 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:13.404870 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-nx96n" podStartSLOduration=2.40484303 podStartE2EDuration="2.40484303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:04:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:04:13.403496252 +0000 UTC m=+262.586340484" watchObservedRunningTime="2026-07-13 22:04:13.40484303 +0000 UTC m=+262.587687260" Jul 13 22:04:23.515518 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:23.515483 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:23.619729 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:23.619694 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:29.350655 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:29.350620 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:29.450624 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:29.450589 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:31.473541 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:31.473504 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:44.937582 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:44.937553 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:45.992525 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:45.992492 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g"] Jul 13 22:04:45.994757 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:45.994732 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.001654 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.001520 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 22:04:46.001790 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.001773 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 22:04:46.001938 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.001917 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 22:04:46.002878 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.002837 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 22:04:46.002972 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.002922 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kwtxt\"" Jul 13 22:04:46.034981 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.034956 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g"] Jul 13 22:04:46.132064 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.132040 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5klx\" (UniqueName: \"kubernetes.io/projected/05edc547-1871-4cdc-8770-d18c09e2dba2-kube-api-access-s5klx\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.132204 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.132077 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/05edc547-1871-4cdc-8770-d18c09e2dba2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.132204 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.132097 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/05edc547-1871-4cdc-8770-d18c09e2dba2-manifests\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.132204 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.132120 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/05edc547-1871-4cdc-8770-d18c09e2dba2-webhook-cert\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.132204 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.132138 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05edc547-1871-4cdc-8770-d18c09e2dba2-tmp\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.233104 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.233076 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5klx\" (UniqueName: \"kubernetes.io/projected/05edc547-1871-4cdc-8770-d18c09e2dba2-kube-api-access-s5klx\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.233248 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.233116 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/05edc547-1871-4cdc-8770-d18c09e2dba2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.233248 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.233136 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/05edc547-1871-4cdc-8770-d18c09e2dba2-manifests\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.233248 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.233158 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/05edc547-1871-4cdc-8770-d18c09e2dba2-webhook-cert\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.233248 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.233180 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05edc547-1871-4cdc-8770-d18c09e2dba2-tmp\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.233557 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.233536 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/05edc547-1871-4cdc-8770-d18c09e2dba2-manifests\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.235450 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.235420 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05edc547-1871-4cdc-8770-d18c09e2dba2-tmp\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.235625 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.235604 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/05edc547-1871-4cdc-8770-d18c09e2dba2-webhook-cert\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.235698 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.235681 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/05edc547-1871-4cdc-8770-d18c09e2dba2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.242466 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.242442 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5klx\" (UniqueName: \"kubernetes.io/projected/05edc547-1871-4cdc-8770-d18c09e2dba2-kube-api-access-s5klx\") pod \"opendatahub-operator-controller-manager-67ddb65fc-vn66g\" (UID: \"05edc547-1871-4cdc-8770-d18c09e2dba2\") " pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.290327 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.290264 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:46.305189 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.305168 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:46.383941 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.383870 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:46.447599 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.447568 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g"] Jul 13 22:04:46.450749 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:04:46.450722 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05edc547_1871_4cdc_8770_d18c09e2dba2.slice/crio-54108cfb034c48f5008d2e1651c40fcfee82a476f9d1054e522759852bcc3c25 WatchSource:0}: Error finding container 54108cfb034c48f5008d2e1651c40fcfee82a476f9d1054e522759852bcc3c25: Status 404 returned error can't find the container with id 54108cfb034c48f5008d2e1651c40fcfee82a476f9d1054e522759852bcc3c25 Jul 13 22:04:46.456658 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.456622 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:46.525617 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.525588 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" event={"ID":"05edc547-1871-4cdc-8770-d18c09e2dba2","Type":"ContainerStarted","Data":"54108cfb034c48f5008d2e1651c40fcfee82a476f9d1054e522759852bcc3c25"} Jul 13 22:04:46.538615 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.538587 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:46.638497 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.638462 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:46.769120 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.769085 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:46.865583 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:46.865557 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:47.880540 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:47.880507 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:47.982328 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:47.982234 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:49.540622 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:49.540589 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" event={"ID":"05edc547-1871-4cdc-8770-d18c09e2dba2","Type":"ContainerStarted","Data":"72febbe94deb6404d32662d8d2a7108ade5f4b1e93f87912377bd0fd1916cd87"} Jul 13 22:04:50.545811 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:50.545787 2570 generic.go:356] "Generic (PLEG): container finished" podID="05edc547-1871-4cdc-8770-d18c09e2dba2" containerID="72febbe94deb6404d32662d8d2a7108ade5f4b1e93f87912377bd0fd1916cd87" exitCode=0 Jul 13 22:04:50.546104 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:50.545838 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" event={"ID":"05edc547-1871-4cdc-8770-d18c09e2dba2","Type":"ContainerDied","Data":"72febbe94deb6404d32662d8d2a7108ade5f4b1e93f87912377bd0fd1916cd87"} Jul 13 22:04:51.187648 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:51.187616 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:04:51.187993 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:51.187969 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:04:51.196024 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:51.196001 2570 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 22:04:51.552901 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:51.552845 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" event={"ID":"05edc547-1871-4cdc-8770-d18c09e2dba2","Type":"ContainerStarted","Data":"6e18d1554506c04319e1b8e266a39175b037b380cd63acfbe7d4d9a90e0f10a4"} Jul 13 22:04:51.555256 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:51.552999 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:04:51.613409 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:51.613355 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" podStartSLOduration=2.177033546 podStartE2EDuration="6.613328598s" totalImagesPullingTime="3.427982992s" totalInitContainerRuntime="1.00831206s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:04:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:04:51.611120622 +0000 UTC m=+300.793964844" watchObservedRunningTime="2026-07-13 22:04:51.613328598 +0000 UTC m=+300.796172819" Jul 13 22:04:57.551041 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:57.551008 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:57.665042 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:04:57.665013 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:02.558658 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:02.558630 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67ddb65fc-vn66g" Jul 13 22:05:04.314762 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.314707 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8"] Jul 13 22:05:04.323334 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.323311 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.325879 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.325841 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 13 22:05:04.325879 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.325843 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 13 22:05:04.326171 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.326156 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 22:05:04.326377 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.326360 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 22:05:04.326430 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.326393 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kv7xp\"" Jul 13 22:05:04.326481 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.326399 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 13 22:05:04.329905 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.329882 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8"] Jul 13 22:05:04.490045 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.490009 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2648c485-2db2-4d6b-9e35-8f9be182b7d6-manager-config\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.490216 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.490054 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr5rw\" (UniqueName: \"kubernetes.io/projected/2648c485-2db2-4d6b-9e35-8f9be182b7d6-kube-api-access-tr5rw\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.490216 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.490168 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2648c485-2db2-4d6b-9e35-8f9be182b7d6-metrics-cert\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.490301 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.490275 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2648c485-2db2-4d6b-9e35-8f9be182b7d6-cert\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.590729 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.590640 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2648c485-2db2-4d6b-9e35-8f9be182b7d6-manager-config\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.590729 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.590690 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr5rw\" (UniqueName: \"kubernetes.io/projected/2648c485-2db2-4d6b-9e35-8f9be182b7d6-kube-api-access-tr5rw\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.590990 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.590737 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2648c485-2db2-4d6b-9e35-8f9be182b7d6-metrics-cert\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.590990 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.590766 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2648c485-2db2-4d6b-9e35-8f9be182b7d6-cert\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.591354 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.591326 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2648c485-2db2-4d6b-9e35-8f9be182b7d6-manager-config\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.593152 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.593122 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2648c485-2db2-4d6b-9e35-8f9be182b7d6-cert\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.593385 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.593362 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2648c485-2db2-4d6b-9e35-8f9be182b7d6-metrics-cert\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.600633 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.600609 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr5rw\" (UniqueName: \"kubernetes.io/projected/2648c485-2db2-4d6b-9e35-8f9be182b7d6-kube-api-access-tr5rw\") pod \"lws-controller-manager-88cc59766-7b2r8\" (UID: \"2648c485-2db2-4d6b-9e35-8f9be182b7d6\") " pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.637018 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.636985 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:04.775138 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.775112 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8"] Jul 13 22:05:04.777583 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:05:04.777557 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2648c485_2db2_4d6b_9e35_8f9be182b7d6.slice/crio-1fa00a997b8365f6e622d475156c10f7e88beeccee8511d24a3130f00fa015a5 WatchSource:0}: Error finding container 1fa00a997b8365f6e622d475156c10f7e88beeccee8511d24a3130f00fa015a5: Status 404 returned error can't find the container with id 1fa00a997b8365f6e622d475156c10f7e88beeccee8511d24a3130f00fa015a5 Jul 13 22:05:04.779508 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.779493 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 22:05:04.966484 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:04.966447 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:05.102027 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:05.101998 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:05.537436 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:05.537200 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:05.612330 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:05.612288 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" event={"ID":"2648c485-2db2-4d6b-9e35-8f9be182b7d6","Type":"ContainerStarted","Data":"1fa00a997b8365f6e622d475156c10f7e88beeccee8511d24a3130f00fa015a5"} Jul 13 22:05:05.897499 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:05.897357 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:06.368814 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:06.368776 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:06.557151 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:06.557121 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:06.744156 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:06.744075 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:07.014335 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:07.014232 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:07.623863 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:07.623822 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" event={"ID":"2648c485-2db2-4d6b-9e35-8f9be182b7d6","Type":"ContainerStarted","Data":"cc79d67eeb6a3a800e9eb08533c7849a4455e767785eb88bf60015cb961dd76f"} Jul 13 22:05:07.624527 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:07.624408 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:07.658659 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:07.658603 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" podStartSLOduration=1.230124366 podStartE2EDuration="3.65858487s" totalImagesPullingTime="2.428460504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:05:07.657876309 +0000 UTC m=+316.840720541" watchObservedRunningTime="2026-07-13 22:05:07.65858487 +0000 UTC m=+316.841429115" Jul 13 22:05:08.190812 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:08.190775 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:08.669823 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:08.669786 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:08.754774 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:08.754733 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:09.854348 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:09.854316 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:14.083371 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.083340 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:14.140159 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.140128 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs"] Jul 13 22:05:14.142192 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.142173 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.144975 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.144954 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 22:05:14.145081 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.144976 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 13 22:05:14.145081 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.145017 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 13 22:05:14.145081 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.144981 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 22:05:14.145592 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.145577 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 22:05:14.145592 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.145583 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 13 22:05:14.156694 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.156678 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6wdkm\"" Jul 13 22:05:14.165735 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.165713 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs"] Jul 13 22:05:14.166453 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.166432 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.166552 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.166473 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3071b09b-3802-4a16-b299-2e04458b8ecd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.166552 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.166488 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.166660 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.166615 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.166715 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.166655 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3071b09b-3802-4a16-b299-2e04458b8ecd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.166715 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.166699 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvcrf\" (UniqueName: \"kubernetes.io/projected/3071b09b-3802-4a16-b299-2e04458b8ecd-kube-api-access-tvcrf\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.166804 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.166729 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.210313 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.210284 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:14.267356 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.267324 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3071b09b-3802-4a16-b299-2e04458b8ecd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.267356 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.267357 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.267570 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.267394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.267570 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.267411 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3071b09b-3802-4a16-b299-2e04458b8ecd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.267570 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.267428 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvcrf\" (UniqueName: \"kubernetes.io/projected/3071b09b-3802-4a16-b299-2e04458b8ecd-kube-api-access-tvcrf\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.267570 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.267449 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.267570 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.267470 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.268084 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.268060 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.270037 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.270013 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.270236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.270208 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.270604 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.270581 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3071b09b-3802-4a16-b299-2e04458b8ecd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.270604 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.270596 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3071b09b-3802-4a16-b299-2e04458b8ecd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.276228 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.276206 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3071b09b-3802-4a16-b299-2e04458b8ecd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.278047 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.278028 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvcrf\" (UniqueName: \"kubernetes.io/projected/3071b09b-3802-4a16-b299-2e04458b8ecd-kube-api-access-tvcrf\") pod \"istiod-openshift-gateway-94bb8fbfd-d6ljs\" (UID: \"3071b09b-3802-4a16-b299-2e04458b8ecd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.451807 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.451777 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:14.606691 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.606669 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs"] Jul 13 22:05:14.608315 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:05:14.608283 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3071b09b_3802_4a16_b299_2e04458b8ecd.slice/crio-dc4f70d127590fc1a208fb87912e112b55a6c3eb622fbc211edfe2960061e137 WatchSource:0}: Error finding container dc4f70d127590fc1a208fb87912e112b55a6c3eb622fbc211edfe2960061e137: Status 404 returned error can't find the container with id dc4f70d127590fc1a208fb87912e112b55a6c3eb622fbc211edfe2960061e137 Jul 13 22:05:14.656412 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.656387 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" event={"ID":"3071b09b-3802-4a16-b299-2e04458b8ecd","Type":"ContainerStarted","Data":"dc4f70d127590fc1a208fb87912e112b55a6c3eb622fbc211edfe2960061e137"} Jul 13 22:05:14.996772 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:14.996724 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:15.090718 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:15.090687 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:15.202507 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:15.202468 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:16.909279 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:16.909244 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 22:05:16.909529 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:16.909312 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 22:05:17.673762 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:17.673720 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" event={"ID":"3071b09b-3802-4a16-b299-2e04458b8ecd","Type":"ContainerStarted","Data":"e516590bd2efbe88ae1ec58e0cf234e24f8eadcc7bb307954febded043e18b3d"} Jul 13 22:05:17.674804 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:17.674771 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:17.676627 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:17.676590 2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-d6ljs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 22:05:17.676732 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:17.676678 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" podUID="3071b09b-3802-4a16-b299-2e04458b8ecd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 22:05:17.745578 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:17.745519 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" podStartSLOduration=1.446887534 podStartE2EDuration="3.745487877s" totalImagesPullingTime="2.298600343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:05:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:05:17.743056922 +0000 UTC m=+326.925901165" watchObservedRunningTime="2026-07-13 22:05:17.745487877 +0000 UTC m=+326.928332109" Jul 13 22:05:18.095898 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:18.095788 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:18.218186 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:18.218146 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:18.301436 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:18.301400 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:18.630014 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:18.629982 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-88cc59766-7b2r8" Jul 13 22:05:18.679482 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:18.679454 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d6ljs" Jul 13 22:05:18.809965 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:18.809932 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:19.459983 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:19.459953 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:19.564101 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:19.564068 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:19.881723 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:19.881693 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:19.976126 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:19.976082 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:25.721087 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:05:25.721059 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:06.472894 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:06.472787 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:06.568362 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:06.568322 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:06.655319 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:06.655290 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:07.400983 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:07.400943 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:07.628379 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:07.628342 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:35.670078 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:35.670040 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:39.953580 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:39.953550 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:42.337769 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:42.337729 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:42.539559 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:42.539532 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:43.718974 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:43.718937 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:46.883237 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:46.883203 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:47.275762 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:47.275671 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:47.573993 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:47.573921 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:47.778186 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:47.778147 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:50.341276 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:50.341245 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:51.697049 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:51.697017 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:51.820233 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:51.820204 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:55.441042 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:55.441007 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:55.541891 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:55.541860 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:57.556230 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:57.556185 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:57.657752 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:57.657711 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:58.911670 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:58.911639 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:59.033359 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:06:59.033328 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:02.279297 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:02.279263 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:07.374175 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:07.374132 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:07.515566 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:07.515532 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:09.813211 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:09.813176 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:10.073390 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:10.073305 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:12.652123 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:12.652082 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:12.754521 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:12.754488 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:15.892638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:15.892602 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:16.359976 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:16.359947 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:16.458021 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:16.457990 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:25.202250 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:25.202210 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:25.303614 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:25.303581 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:27.942667 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:27.942633 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:28.094794 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:28.094762 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:31.004234 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:31.004197 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:31.105652 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:31.105610 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:34.643926 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:34.643895 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:34.741509 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:34.741476 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:38.928401 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:38.928356 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:39.009711 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.009672 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:39.103201 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.103160 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:39.220959 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.220887 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69675fd648-7vfh9"] Jul 13 22:07:39.224021 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.224001 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.235347 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.235325 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69675fd648-7vfh9"] Jul 13 22:07:39.288421 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.288381 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:39.317030 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.317010 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/adc89823-d927-4d71-9136-c31d17bed90d-console-oauth-config\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.317149 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.317037 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-oauth-serving-cert\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.317149 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.317064 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-service-ca\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.317149 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.317107 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/adc89823-d927-4d71-9136-c31d17bed90d-console-serving-cert\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.317149 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.317136 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-trusted-ca-bundle\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.317275 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.317178 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-console-config\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.317275 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.317209 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf9m7\" (UniqueName: \"kubernetes.io/projected/adc89823-d927-4d71-9136-c31d17bed90d-kube-api-access-bf9m7\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.363845 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.363819 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:39.418028 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.418001 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-console-config\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.418155 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.418038 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf9m7\" (UniqueName: \"kubernetes.io/projected/adc89823-d927-4d71-9136-c31d17bed90d-kube-api-access-bf9m7\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.418155 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.418069 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/adc89823-d927-4d71-9136-c31d17bed90d-console-oauth-config\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.418155 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.418088 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-oauth-serving-cert\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.418155 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.418135 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-service-ca\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.418363 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.418315 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/adc89823-d927-4d71-9136-c31d17bed90d-console-serving-cert\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.418363 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.418353 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-trusted-ca-bundle\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.418827 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.418806 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-service-ca\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.419023 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.419001 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-console-config\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.419076 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.419001 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-oauth-serving-cert\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.419545 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.419524 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adc89823-d927-4d71-9136-c31d17bed90d-trusted-ca-bundle\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.420689 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.420662 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/adc89823-d927-4d71-9136-c31d17bed90d-console-serving-cert\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.420811 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.420790 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/adc89823-d927-4d71-9136-c31d17bed90d-console-oauth-config\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.427046 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.427024 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf9m7\" (UniqueName: \"kubernetes.io/projected/adc89823-d927-4d71-9136-c31d17bed90d-kube-api-access-bf9m7\") pod \"console-69675fd648-7vfh9\" (UID: \"adc89823-d927-4d71-9136-c31d17bed90d\") " pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.533323 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.533252 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:39.665842 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:39.665816 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69675fd648-7vfh9"] Jul 13 22:07:39.668444 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:07:39.668417 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadc89823_d927_4d71_9136_c31d17bed90d.slice/crio-ecb5cfc0d2a06eb45e3e35a06abaf549439df8c6f2039d91206195e6be058c79 WatchSource:0}: Error finding container ecb5cfc0d2a06eb45e3e35a06abaf549439df8c6f2039d91206195e6be058c79: Status 404 returned error can't find the container with id ecb5cfc0d2a06eb45e3e35a06abaf549439df8c6f2039d91206195e6be058c79 Jul 13 22:07:40.255461 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.255427 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69675fd648-7vfh9" event={"ID":"adc89823-d927-4d71-9136-c31d17bed90d","Type":"ContainerStarted","Data":"1ba3fbe3f52c894b238175fdebdd698d639feb448851b83615dea1694bd54826"} Jul 13 22:07:40.255869 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.255479 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69675fd648-7vfh9" event={"ID":"adc89823-d927-4d71-9136-c31d17bed90d","Type":"ContainerStarted","Data":"ecb5cfc0d2a06eb45e3e35a06abaf549439df8c6f2039d91206195e6be058c79"} Jul 13 22:07:40.284994 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.284953 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69675fd648-7vfh9" podStartSLOduration=1.284938834 podStartE2EDuration="1.284938834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:07:40.282697533 +0000 UTC m=+469.465541763" watchObservedRunningTime="2026-07-13 22:07:40.284938834 +0000 UTC m=+469.467783065" Jul 13 22:07:40.578810 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.578710 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:40.589482 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.589454 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvsvb"] Jul 13 22:07:40.592606 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.592591 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:07:40.594630 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.594609 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 22:07:40.594715 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.594682 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 22:07:40.595098 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.595081 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xnhp8\"" Jul 13 22:07:40.595098 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.595093 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 22:07:40.600544 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.600524 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvsvb"] Jul 13 22:07:40.630574 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.630551 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92z2q\" (UniqueName: \"kubernetes.io/projected/8ad0369c-7a32-45e5-af45-dcaec17da1e5-kube-api-access-92z2q\") pod \"limitador-limitador-64c94fd64f-vvsvb\" (UID: \"8ad0369c-7a32-45e5-af45-dcaec17da1e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:07:40.630673 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.630610 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8ad0369c-7a32-45e5-af45-dcaec17da1e5-config-file\") pod \"limitador-limitador-64c94fd64f-vvsvb\" (UID: \"8ad0369c-7a32-45e5-af45-dcaec17da1e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:07:40.687952 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.687918 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:40.692950 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.692928 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvsvb"] Jul 13 22:07:40.731470 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.731440 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8ad0369c-7a32-45e5-af45-dcaec17da1e5-config-file\") pod \"limitador-limitador-64c94fd64f-vvsvb\" (UID: \"8ad0369c-7a32-45e5-af45-dcaec17da1e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:07:40.731598 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.731490 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92z2q\" (UniqueName: \"kubernetes.io/projected/8ad0369c-7a32-45e5-af45-dcaec17da1e5-kube-api-access-92z2q\") pod \"limitador-limitador-64c94fd64f-vvsvb\" (UID: \"8ad0369c-7a32-45e5-af45-dcaec17da1e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:07:40.732043 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.732026 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8ad0369c-7a32-45e5-af45-dcaec17da1e5-config-file\") pod \"limitador-limitador-64c94fd64f-vvsvb\" (UID: \"8ad0369c-7a32-45e5-af45-dcaec17da1e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:07:40.742535 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.742510 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92z2q\" (UniqueName: \"kubernetes.io/projected/8ad0369c-7a32-45e5-af45-dcaec17da1e5-kube-api-access-92z2q\") pod \"limitador-limitador-64c94fd64f-vvsvb\" (UID: \"8ad0369c-7a32-45e5-af45-dcaec17da1e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:07:40.788164 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.788140 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:40.903813 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:40.903791 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:07:41.030284 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:41.030263 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvsvb"] Jul 13 22:07:41.032492 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:07:41.032464 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad0369c_7a32_45e5_af45_dcaec17da1e5.slice/crio-6d7e60a760e4a13e99bf97c22676e098bf9a279bc6efdd6b70a75d22172e6f12 WatchSource:0}: Error finding container 6d7e60a760e4a13e99bf97c22676e098bf9a279bc6efdd6b70a75d22172e6f12: Status 404 returned error can't find the container with id 6d7e60a760e4a13e99bf97c22676e098bf9a279bc6efdd6b70a75d22172e6f12 Jul 13 22:07:41.261238 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:41.261165 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" event={"ID":"8ad0369c-7a32-45e5-af45-dcaec17da1e5","Type":"ContainerStarted","Data":"6d7e60a760e4a13e99bf97c22676e098bf9a279bc6efdd6b70a75d22172e6f12"} Jul 13 22:07:41.474170 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:41.474135 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:41.575803 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:41.575730 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:44.280231 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:44.280195 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" event={"ID":"8ad0369c-7a32-45e5-af45-dcaec17da1e5","Type":"ContainerStarted","Data":"d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba"} Jul 13 22:07:44.280664 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:44.280334 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:07:44.301675 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:44.301627 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" podStartSLOduration=1.67575322 podStartE2EDuration="4.30161411s" totalImagesPullingTime="2.62586089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:07:44.299540375 +0000 UTC m=+473.482384618" watchObservedRunningTime="2026-07-13 22:07:44.30161411 +0000 UTC m=+473.484458340" Jul 13 22:07:44.428505 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:44.428474 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:44.487240 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:44.487206 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:45.014814 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:45.014776 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:45.080387 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:45.080353 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:49.534201 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:49.534173 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:49.534583 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:49.534449 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:49.539182 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:49.539161 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:50.317026 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:50.316997 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69675fd648-7vfh9" Jul 13 22:07:50.370918 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:50.370883 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8ff766d9d-gdrhl"] Jul 13 22:07:50.504275 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:50.504235 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:50.614207 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:50.614171 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:55.286646 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:55.286617 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:07:59.328086 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:59.328042 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvsvb"] Jul 13 22:07:59.328526 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:59.328234 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" podUID="8ad0369c-7a32-45e5-af45-dcaec17da1e5" containerName="limitador" containerID="cri-o://d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba" gracePeriod=30 Jul 13 22:07:59.881029 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:59.881003 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:07:59.999306 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:59.999209 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92z2q\" (UniqueName: \"kubernetes.io/projected/8ad0369c-7a32-45e5-af45-dcaec17da1e5-kube-api-access-92z2q\") pod \"8ad0369c-7a32-45e5-af45-dcaec17da1e5\" (UID: \"8ad0369c-7a32-45e5-af45-dcaec17da1e5\") " Jul 13 22:07:59.999451 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:59.999334 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8ad0369c-7a32-45e5-af45-dcaec17da1e5-config-file\") pod \"8ad0369c-7a32-45e5-af45-dcaec17da1e5\" (UID: \"8ad0369c-7a32-45e5-af45-dcaec17da1e5\") " Jul 13 22:07:59.999690 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:07:59.999671 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ad0369c-7a32-45e5-af45-dcaec17da1e5-config-file" (OuterVolumeSpecName: "config-file") pod "8ad0369c-7a32-45e5-af45-dcaec17da1e5" (UID: "8ad0369c-7a32-45e5-af45-dcaec17da1e5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:08:00.001413 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.001391 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ad0369c-7a32-45e5-af45-dcaec17da1e5-kube-api-access-92z2q" (OuterVolumeSpecName: "kube-api-access-92z2q") pod "8ad0369c-7a32-45e5-af45-dcaec17da1e5" (UID: "8ad0369c-7a32-45e5-af45-dcaec17da1e5"). InnerVolumeSpecName "kube-api-access-92z2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:08:00.100143 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.100111 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92z2q\" (UniqueName: \"kubernetes.io/projected/8ad0369c-7a32-45e5-af45-dcaec17da1e5-kube-api-access-92z2q\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:08:00.100143 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.100139 2570 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8ad0369c-7a32-45e5-af45-dcaec17da1e5-config-file\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:08:00.356023 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.355938 2570 generic.go:356] "Generic (PLEG): container finished" podID="8ad0369c-7a32-45e5-af45-dcaec17da1e5" containerID="d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba" exitCode=0 Jul 13 22:08:00.356023 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.355983 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" Jul 13 22:08:00.356574 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.356027 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" event={"ID":"8ad0369c-7a32-45e5-af45-dcaec17da1e5","Type":"ContainerDied","Data":"d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba"} Jul 13 22:08:00.356574 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.356083 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvsvb" event={"ID":"8ad0369c-7a32-45e5-af45-dcaec17da1e5","Type":"ContainerDied","Data":"6d7e60a760e4a13e99bf97c22676e098bf9a279bc6efdd6b70a75d22172e6f12"} Jul 13 22:08:00.356574 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.356116 2570 scope.go:117] "RemoveContainer" containerID="d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba" Jul 13 22:08:00.364813 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.364794 2570 scope.go:117] "RemoveContainer" containerID="d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba" Jul 13 22:08:00.365081 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:08:00.365062 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba\": container with ID starting with d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba not found: ID does not exist" containerID="d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba" Jul 13 22:08:00.365156 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.365092 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba"} err="failed to get container status \"d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba\": rpc error: code = NotFound desc = could not find container \"d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba\": container with ID starting with d462a5d0775ed987819594a009f8bfa0f272186eebb4b52552976070799c7aba not found: ID does not exist" Jul 13 22:08:00.377586 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.377560 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvsvb"] Jul 13 22:08:00.379694 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:00.379670 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvsvb"] Jul 13 22:08:01.316621 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:01.316589 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ad0369c-7a32-45e5-af45-dcaec17da1e5" path="/var/lib/kubelet/pods/8ad0369c-7a32-45e5-af45-dcaec17da1e5/volumes" Jul 13 22:08:01.875229 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:01.875188 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:01.989247 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:01.989217 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:02.378296 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.378261 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8khfd"] Jul 13 22:08:02.378621 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.378608 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad0369c-7a32-45e5-af45-dcaec17da1e5" containerName="limitador" Jul 13 22:08:02.378665 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.378623 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad0369c-7a32-45e5-af45-dcaec17da1e5" containerName="limitador" Jul 13 22:08:02.378702 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.378682 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ad0369c-7a32-45e5-af45-dcaec17da1e5" containerName="limitador" Jul 13 22:08:02.382736 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.382719 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" Jul 13 22:08:02.385490 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.385465 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 22:08:02.385906 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.385891 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 13 22:08:02.385906 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.385897 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-plhwl\"" Jul 13 22:08:02.386043 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.385929 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 22:08:02.400670 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.400642 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8khfd"] Jul 13 22:08:02.407890 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.407861 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:02.522474 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.522431 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/10d24f67-b1cc-4b6b-a6d0-320b0f4b957a-data\") pod \"postgres-69fb4594cf-8khfd\" (UID: \"10d24f67-b1cc-4b6b-a6d0-320b0f4b957a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" Jul 13 22:08:02.522474 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.522469 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpj5r\" (UniqueName: \"kubernetes.io/projected/10d24f67-b1cc-4b6b-a6d0-320b0f4b957a-kube-api-access-zpj5r\") pod \"postgres-69fb4594cf-8khfd\" (UID: \"10d24f67-b1cc-4b6b-a6d0-320b0f4b957a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" Jul 13 22:08:02.623427 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.623393 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/10d24f67-b1cc-4b6b-a6d0-320b0f4b957a-data\") pod \"postgres-69fb4594cf-8khfd\" (UID: \"10d24f67-b1cc-4b6b-a6d0-320b0f4b957a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" Jul 13 22:08:02.623427 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.623426 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpj5r\" (UniqueName: \"kubernetes.io/projected/10d24f67-b1cc-4b6b-a6d0-320b0f4b957a-kube-api-access-zpj5r\") pod \"postgres-69fb4594cf-8khfd\" (UID: \"10d24f67-b1cc-4b6b-a6d0-320b0f4b957a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" Jul 13 22:08:02.623794 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.623775 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/10d24f67-b1cc-4b6b-a6d0-320b0f4b957a-data\") pod \"postgres-69fb4594cf-8khfd\" (UID: \"10d24f67-b1cc-4b6b-a6d0-320b0f4b957a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" Jul 13 22:08:02.636120 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.636051 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpj5r\" (UniqueName: \"kubernetes.io/projected/10d24f67-b1cc-4b6b-a6d0-320b0f4b957a-kube-api-access-zpj5r\") pod \"postgres-69fb4594cf-8khfd\" (UID: \"10d24f67-b1cc-4b6b-a6d0-320b0f4b957a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" Jul 13 22:08:02.694112 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.694075 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" Jul 13 22:08:02.823056 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:02.822935 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8khfd"] Jul 13 22:08:02.825381 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:08:02.825356 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d24f67_b1cc_4b6b_a6d0_320b0f4b957a.slice/crio-220ecc1fba4e4f2be082b13ce7d964f1d8d8efac672f417ce25a56a6a4666e53 WatchSource:0}: Error finding container 220ecc1fba4e4f2be082b13ce7d964f1d8d8efac672f417ce25a56a6a4666e53: Status 404 returned error can't find the container with id 220ecc1fba4e4f2be082b13ce7d964f1d8d8efac672f417ce25a56a6a4666e53 Jul 13 22:08:03.371022 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:03.370987 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" event={"ID":"10d24f67-b1cc-4b6b-a6d0-320b0f4b957a","Type":"ContainerStarted","Data":"220ecc1fba4e4f2be082b13ce7d964f1d8d8efac672f417ce25a56a6a4666e53"} Jul 13 22:08:09.400972 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:09.400940 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" event={"ID":"10d24f67-b1cc-4b6b-a6d0-320b0f4b957a","Type":"ContainerStarted","Data":"905e8cd659ae3df0af68e98e7579ee2dba5cc47a90bd6a15d049a17610ecff57"} Jul 13 22:08:09.401340 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:09.401014 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" Jul 13 22:08:09.416433 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:09.416373 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" podStartSLOduration=1.528427303 podStartE2EDuration="7.416354964s" totalImagesPullingTime="5.887927661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:08:09.416233417 +0000 UTC m=+498.599077662" watchObservedRunningTime="2026-07-13 22:08:09.416354964 +0000 UTC m=+498.599199199" Jul 13 22:08:15.398313 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.398265 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8ff766d9d-gdrhl" podUID="9494c095-f074-45f3-b8a2-5154f589add4" containerName="console" containerID="cri-o://185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5" gracePeriod=15 Jul 13 22:08:15.441569 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.441543 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8khfd" Jul 13 22:08:15.636486 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.636463 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:08:15.744030 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.743956 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-serving-cert\") pod \"9494c095-f074-45f3-b8a2-5154f589add4\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " Jul 13 22:08:15.744030 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.744005 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-oauth-serving-cert\") pod \"9494c095-f074-45f3-b8a2-5154f589add4\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " Jul 13 22:08:15.744242 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.744037 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-oauth-config\") pod \"9494c095-f074-45f3-b8a2-5154f589add4\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " Jul 13 22:08:15.744242 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.744053 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-console-config\") pod \"9494c095-f074-45f3-b8a2-5154f589add4\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " Jul 13 22:08:15.744242 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.744104 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-trusted-ca-bundle\") pod \"9494c095-f074-45f3-b8a2-5154f589add4\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " Jul 13 22:08:15.744242 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.744226 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-service-ca\") pod \"9494c095-f074-45f3-b8a2-5154f589add4\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " Jul 13 22:08:15.744444 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.744275 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28hfr\" (UniqueName: \"kubernetes.io/projected/9494c095-f074-45f3-b8a2-5154f589add4-kube-api-access-28hfr\") pod \"9494c095-f074-45f3-b8a2-5154f589add4\" (UID: \"9494c095-f074-45f3-b8a2-5154f589add4\") " Jul 13 22:08:15.744556 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.744533 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9494c095-f074-45f3-b8a2-5154f589add4" (UID: "9494c095-f074-45f3-b8a2-5154f589add4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:08:15.744556 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.744546 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-console-config" (OuterVolumeSpecName: "console-config") pod "9494c095-f074-45f3-b8a2-5154f589add4" (UID: "9494c095-f074-45f3-b8a2-5154f589add4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:08:15.744678 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.744591 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9494c095-f074-45f3-b8a2-5154f589add4" (UID: "9494c095-f074-45f3-b8a2-5154f589add4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:08:15.744678 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.744608 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-service-ca" (OuterVolumeSpecName: "service-ca") pod "9494c095-f074-45f3-b8a2-5154f589add4" (UID: "9494c095-f074-45f3-b8a2-5154f589add4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:08:15.746466 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.746434 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9494c095-f074-45f3-b8a2-5154f589add4-kube-api-access-28hfr" (OuterVolumeSpecName: "kube-api-access-28hfr") pod "9494c095-f074-45f3-b8a2-5154f589add4" (UID: "9494c095-f074-45f3-b8a2-5154f589add4"). InnerVolumeSpecName "kube-api-access-28hfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:08:15.746466 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.746450 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9494c095-f074-45f3-b8a2-5154f589add4" (UID: "9494c095-f074-45f3-b8a2-5154f589add4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:08:15.746649 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.746496 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9494c095-f074-45f3-b8a2-5154f589add4" (UID: "9494c095-f074-45f3-b8a2-5154f589add4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:08:15.779641 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.779606 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:15.845021 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.845001 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-oauth-config\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:08:15.845021 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.845021 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-console-config\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:08:15.845166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.845032 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-trusted-ca-bundle\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:08:15.845166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.845041 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:08:15.845166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.845050 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28hfr\" (UniqueName: \"kubernetes.io/projected/9494c095-f074-45f3-b8a2-5154f589add4-kube-api-access-28hfr\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:08:15.845166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.845058 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9494c095-f074-45f3-b8a2-5154f589add4-console-serving-cert\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:08:15.845166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.845066 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9494c095-f074-45f3-b8a2-5154f589add4-oauth-serving-cert\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:08:15.911644 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:15.911610 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:16.437612 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:16.437579 2570 generic.go:356] "Generic (PLEG): container finished" podID="9494c095-f074-45f3-b8a2-5154f589add4" containerID="185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5" exitCode=0 Jul 13 22:08:16.438010 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:16.437629 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8ff766d9d-gdrhl" Jul 13 22:08:16.438010 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:16.437666 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8ff766d9d-gdrhl" event={"ID":"9494c095-f074-45f3-b8a2-5154f589add4","Type":"ContainerDied","Data":"185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5"} Jul 13 22:08:16.438010 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:16.437743 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8ff766d9d-gdrhl" event={"ID":"9494c095-f074-45f3-b8a2-5154f589add4","Type":"ContainerDied","Data":"56a3ccc988e79b719582dd9d88c1424bc59228ed26ee8fed0d8c5fc5f9f4ea21"} Jul 13 22:08:16.438010 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:16.437769 2570 scope.go:117] "RemoveContainer" containerID="185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5" Jul 13 22:08:16.446482 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:16.446466 2570 scope.go:117] "RemoveContainer" containerID="185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5" Jul 13 22:08:16.446720 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:08:16.446702 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5\": container with ID starting with 185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5 not found: ID does not exist" containerID="185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5" Jul 13 22:08:16.446786 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:16.446731 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5"} err="failed to get container status \"185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5\": rpc error: code = NotFound desc = could not find container \"185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5\": container with ID starting with 185780455f9476710a18b6b2158becc2e98deed0ed66f0018e11a9763b400ef5 not found: ID does not exist" Jul 13 22:08:16.460821 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:16.460799 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8ff766d9d-gdrhl"] Jul 13 22:08:16.464629 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:16.464608 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8ff766d9d-gdrhl"] Jul 13 22:08:17.316207 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:17.316171 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9494c095-f074-45f3-b8a2-5154f589add4" path="/var/lib/kubelet/pods/9494c095-f074-45f3-b8a2-5154f589add4/volumes" Jul 13 22:08:22.513462 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.513417 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:22.518517 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.518491 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79f5888f6-4nwgm"] Jul 13 22:08:22.518829 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.518816 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9494c095-f074-45f3-b8a2-5154f589add4" containerName="console" Jul 13 22:08:22.518890 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.518831 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9494c095-f074-45f3-b8a2-5154f589add4" containerName="console" Jul 13 22:08:22.518927 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.518907 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9494c095-f074-45f3-b8a2-5154f589add4" containerName="console" Jul 13 22:08:22.545925 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.545881 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79f5888f6-4nwgm"] Jul 13 22:08:22.546104 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.545995 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:22.548612 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.548583 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 22:08:22.548612 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.548610 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 22:08:22.548808 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.548631 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d99t2\"" Jul 13 22:08:22.583029 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.582990 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:22.592175 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.592150 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mssst\" (UniqueName: \"kubernetes.io/projected/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-kube-api-access-mssst\") pod \"maas-controller-79f5888f6-4nwgm\" (UID: \"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3\") " pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:22.592251 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.592194 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-webhook-cert\") pod \"maas-controller-79f5888f6-4nwgm\" (UID: \"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3\") " pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:22.649366 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.649329 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:22.693123 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.693085 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mssst\" (UniqueName: \"kubernetes.io/projected/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-kube-api-access-mssst\") pod \"maas-controller-79f5888f6-4nwgm\" (UID: \"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3\") " pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:22.693300 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.693130 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-webhook-cert\") pod \"maas-controller-79f5888f6-4nwgm\" (UID: \"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3\") " pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:22.693300 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:08:22.693239 2570 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 13 22:08:22.693410 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:08:22.693303 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-webhook-cert podName:67dd9cc6-87ea-4ef6-bc41-8595f3e231e3 nodeName:}" failed. No retries permitted until 2026-07-13 22:08:23.193284476 +0000 UTC m=+512.376128708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-webhook-cert") pod "maas-controller-79f5888f6-4nwgm" (UID: "67dd9cc6-87ea-4ef6-bc41-8595f3e231e3") : secret "maas-controller-webhook-cert" not found Jul 13 22:08:22.711815 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.711782 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6dc46d8554-p2sgf"] Jul 13 22:08:22.712825 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.712800 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mssst\" (UniqueName: \"kubernetes.io/projected/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-kube-api-access-mssst\") pod \"maas-controller-79f5888f6-4nwgm\" (UID: \"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3\") " pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:22.715127 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.715109 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:08:22.735112 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.735090 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dc46d8554-p2sgf"] Jul 13 22:08:22.794147 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.794069 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w4qm\" (UniqueName: \"kubernetes.io/projected/d8d8470f-478f-4a36-8b90-de6504a8a76a-kube-api-access-9w4qm\") pod \"maas-controller-6dc46d8554-p2sgf\" (UID: \"d8d8470f-478f-4a36-8b90-de6504a8a76a\") " pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:08:22.794147 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.794121 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8d8470f-478f-4a36-8b90-de6504a8a76a-webhook-cert\") pod \"maas-controller-6dc46d8554-p2sgf\" (UID: \"d8d8470f-478f-4a36-8b90-de6504a8a76a\") " pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:08:22.868154 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.868121 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:22.895190 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.895162 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4qm\" (UniqueName: \"kubernetes.io/projected/d8d8470f-478f-4a36-8b90-de6504a8a76a-kube-api-access-9w4qm\") pod \"maas-controller-6dc46d8554-p2sgf\" (UID: \"d8d8470f-478f-4a36-8b90-de6504a8a76a\") " pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:08:22.895315 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.895204 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8d8470f-478f-4a36-8b90-de6504a8a76a-webhook-cert\") pod \"maas-controller-6dc46d8554-p2sgf\" (UID: \"d8d8470f-478f-4a36-8b90-de6504a8a76a\") " pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:08:22.897498 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.897477 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8d8470f-478f-4a36-8b90-de6504a8a76a-webhook-cert\") pod \"maas-controller-6dc46d8554-p2sgf\" (UID: \"d8d8470f-478f-4a36-8b90-de6504a8a76a\") " pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:08:22.908675 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:22.908652 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w4qm\" (UniqueName: \"kubernetes.io/projected/d8d8470f-478f-4a36-8b90-de6504a8a76a-kube-api-access-9w4qm\") pod \"maas-controller-6dc46d8554-p2sgf\" (UID: \"d8d8470f-478f-4a36-8b90-de6504a8a76a\") " pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:08:23.028435 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:23.028406 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:08:23.164518 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:23.164428 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dc46d8554-p2sgf"] Jul 13 22:08:23.167066 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:08:23.167041 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8d8470f_478f_4a36_8b90_de6504a8a76a.slice/crio-12feb823812595195210e9577ad28f250441bc3d410faa6ae450b6c42b30edf0 WatchSource:0}: Error finding container 12feb823812595195210e9577ad28f250441bc3d410faa6ae450b6c42b30edf0: Status 404 returned error can't find the container with id 12feb823812595195210e9577ad28f250441bc3d410faa6ae450b6c42b30edf0 Jul 13 22:08:23.197949 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:23.197918 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-webhook-cert\") pod \"maas-controller-79f5888f6-4nwgm\" (UID: \"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3\") " pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:23.200187 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:23.200169 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-webhook-cert\") pod \"maas-controller-79f5888f6-4nwgm\" (UID: \"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3\") " pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:23.457556 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:23.457522 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:23.472202 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:23.472175 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" event={"ID":"d8d8470f-478f-4a36-8b90-de6504a8a76a","Type":"ContainerStarted","Data":"12feb823812595195210e9577ad28f250441bc3d410faa6ae450b6c42b30edf0"} Jul 13 22:08:23.588316 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:23.588291 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79f5888f6-4nwgm"] Jul 13 22:08:23.589221 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:08:23.589192 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67dd9cc6_87ea_4ef6_bc41_8595f3e231e3.slice/crio-bd8886ff591280df31f034f20bb89e4a333c6a936a86d7b87f70e7d3e4780aa5 WatchSource:0}: Error finding container bd8886ff591280df31f034f20bb89e4a333c6a936a86d7b87f70e7d3e4780aa5: Status 404 returned error can't find the container with id bd8886ff591280df31f034f20bb89e4a333c6a936a86d7b87f70e7d3e4780aa5 Jul 13 22:08:24.479218 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:24.479162 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f5888f6-4nwgm" event={"ID":"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3","Type":"ContainerStarted","Data":"bd8886ff591280df31f034f20bb89e4a333c6a936a86d7b87f70e7d3e4780aa5"} Jul 13 22:08:25.332328 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:25.332283 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:25.431260 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:25.431227 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:25.488691 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:25.488656 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" event={"ID":"d8d8470f-478f-4a36-8b90-de6504a8a76a","Type":"ContainerStarted","Data":"58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078"} Jul 13 22:08:25.488904 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:25.488768 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:08:25.490038 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:25.490014 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f5888f6-4nwgm" event={"ID":"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3","Type":"ContainerStarted","Data":"04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101"} Jul 13 22:08:25.490133 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:25.490113 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:25.511473 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:25.511427 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" podStartSLOduration=1.539589475 podStartE2EDuration="3.511413106s" totalImagesPullingTime="1.971823631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:08:25.509774928 +0000 UTC m=+514.692619158" watchObservedRunningTime="2026-07-13 22:08:25.511413106 +0000 UTC m=+514.694257339" Jul 13 22:08:25.530421 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:25.530370 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79f5888f6-4nwgm" podStartSLOduration=1.766807561 podStartE2EDuration="3.530353443s" totalImagesPullingTime="1.763545882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:08:25.527981295 +0000 UTC m=+514.710825526" watchObservedRunningTime="2026-07-13 22:08:25.530353443 +0000 UTC m=+514.713197675" Jul 13 22:08:36.497052 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:36.497022 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:36.497424 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:36.497098 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:08:36.564901 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:36.564867 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79f5888f6-4nwgm"] Jul 13 22:08:36.565077 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:36.565057 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-79f5888f6-4nwgm" podUID="67dd9cc6-87ea-4ef6-bc41-8595f3e231e3" containerName="manager" containerID="cri-o://04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101" gracePeriod=30 Jul 13 22:08:36.806169 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:36.806142 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:36.903638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:36.903602 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-webhook-cert\") pod \"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3\" (UID: \"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3\") " Jul 13 22:08:36.903820 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:36.903659 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mssst\" (UniqueName: \"kubernetes.io/projected/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-kube-api-access-mssst\") pod \"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3\" (UID: \"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3\") " Jul 13 22:08:36.905896 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:36.905863 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "67dd9cc6-87ea-4ef6-bc41-8595f3e231e3" (UID: "67dd9cc6-87ea-4ef6-bc41-8595f3e231e3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:08:36.905896 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:36.905869 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-kube-api-access-mssst" (OuterVolumeSpecName: "kube-api-access-mssst") pod "67dd9cc6-87ea-4ef6-bc41-8595f3e231e3" (UID: "67dd9cc6-87ea-4ef6-bc41-8595f3e231e3"). InnerVolumeSpecName "kube-api-access-mssst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:08:37.004412 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:37.004361 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-webhook-cert\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:08:37.004412 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:37.004406 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mssst\" (UniqueName: \"kubernetes.io/projected/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3-kube-api-access-mssst\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:08:37.545305 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:37.545270 2570 generic.go:356] "Generic (PLEG): container finished" podID="67dd9cc6-87ea-4ef6-bc41-8595f3e231e3" containerID="04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101" exitCode=0 Jul 13 22:08:37.545756 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:37.545312 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f5888f6-4nwgm" Jul 13 22:08:37.545756 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:37.545314 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f5888f6-4nwgm" event={"ID":"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3","Type":"ContainerDied","Data":"04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101"} Jul 13 22:08:37.545756 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:37.545454 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f5888f6-4nwgm" event={"ID":"67dd9cc6-87ea-4ef6-bc41-8595f3e231e3","Type":"ContainerDied","Data":"bd8886ff591280df31f034f20bb89e4a333c6a936a86d7b87f70e7d3e4780aa5"} Jul 13 22:08:37.545756 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:37.545481 2570 scope.go:117] "RemoveContainer" containerID="04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101" Jul 13 22:08:37.553564 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:37.553547 2570 scope.go:117] "RemoveContainer" containerID="04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101" Jul 13 22:08:37.553813 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:08:37.553796 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101\": container with ID starting with 04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101 not found: ID does not exist" containerID="04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101" Jul 13 22:08:37.553872 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:37.553828 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101"} err="failed to get container status \"04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101\": rpc error: code = NotFound desc = could not find container \"04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101\": container with ID starting with 04b763cc9e7531db1bef35ed531955aaf84116aac5228a3459961296a3b4a101 not found: ID does not exist" Jul 13 22:08:37.564966 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:37.562034 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79f5888f6-4nwgm"] Jul 13 22:08:37.567675 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:37.567656 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79f5888f6-4nwgm"] Jul 13 22:08:39.316385 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:39.316346 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67dd9cc6-87ea-4ef6-bc41-8595f3e231e3" path="/var/lib/kubelet/pods/67dd9cc6-87ea-4ef6-bc41-8595f3e231e3/volumes" Jul 13 22:08:56.913119 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:56.913062 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:57.006081 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.006048 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:57.513182 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.513148 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:57.615656 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.615624 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz"] Jul 13 22:08:57.615961 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.615949 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67dd9cc6-87ea-4ef6-bc41-8595f3e231e3" containerName="manager" Jul 13 22:08:57.616020 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.615963 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="67dd9cc6-87ea-4ef6-bc41-8595f3e231e3" containerName="manager" Jul 13 22:08:57.616020 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.616016 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="67dd9cc6-87ea-4ef6-bc41-8595f3e231e3" containerName="manager" Jul 13 22:08:57.621553 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.621534 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:08:57.625428 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.625405 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 13 22:08:57.625717 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.625703 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pqthg\"" Jul 13 22:08:57.638644 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.638622 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz"] Jul 13 22:08:57.771787 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.771694 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:57.778692 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.778659 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrddl\" (UniqueName: \"kubernetes.io/projected/5ad98b2f-3bf7-4019-9076-361c089c95b5-kube-api-access-hrddl\") pod \"maas-api-6454d8c544-jcfgz\" (UID: \"5ad98b2f-3bf7-4019-9076-361c089c95b5\") " pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:08:57.778825 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.778712 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad98b2f-3bf7-4019-9076-361c089c95b5-openshift-service-ca\") pod \"maas-api-6454d8c544-jcfgz\" (UID: \"5ad98b2f-3bf7-4019-9076-361c089c95b5\") " pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:08:57.778890 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.778822 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad98b2f-3bf7-4019-9076-361c089c95b5-maas-api-tls\") pod \"maas-api-6454d8c544-jcfgz\" (UID: \"5ad98b2f-3bf7-4019-9076-361c089c95b5\") " pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:08:57.880152 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.880112 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad98b2f-3bf7-4019-9076-361c089c95b5-maas-api-tls\") pod \"maas-api-6454d8c544-jcfgz\" (UID: \"5ad98b2f-3bf7-4019-9076-361c089c95b5\") " pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:08:57.880152 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.880160 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrddl\" (UniqueName: \"kubernetes.io/projected/5ad98b2f-3bf7-4019-9076-361c089c95b5-kube-api-access-hrddl\") pod \"maas-api-6454d8c544-jcfgz\" (UID: \"5ad98b2f-3bf7-4019-9076-361c089c95b5\") " pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:08:57.880363 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.880185 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad98b2f-3bf7-4019-9076-361c089c95b5-openshift-service-ca\") pod \"maas-api-6454d8c544-jcfgz\" (UID: \"5ad98b2f-3bf7-4019-9076-361c089c95b5\") " pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:08:57.880777 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.880761 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad98b2f-3bf7-4019-9076-361c089c95b5-openshift-service-ca\") pod \"maas-api-6454d8c544-jcfgz\" (UID: \"5ad98b2f-3bf7-4019-9076-361c089c95b5\") " pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:08:57.882577 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.882555 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad98b2f-3bf7-4019-9076-361c089c95b5-maas-api-tls\") pod \"maas-api-6454d8c544-jcfgz\" (UID: \"5ad98b2f-3bf7-4019-9076-361c089c95b5\") " pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:08:57.892887 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.892868 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrddl\" (UniqueName: \"kubernetes.io/projected/5ad98b2f-3bf7-4019-9076-361c089c95b5-kube-api-access-hrddl\") pod \"maas-api-6454d8c544-jcfgz\" (UID: \"5ad98b2f-3bf7-4019-9076-361c089c95b5\") " pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:08:57.931841 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:57.931819 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:08:58.108294 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:08:58.108268 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad98b2f_3bf7_4019_9076_361c089c95b5.slice/crio-c4b9e13aa4addccf8b99a10e923437495ff74b7b6cec5733079ed3d5565ff7bc WatchSource:0}: Error finding container c4b9e13aa4addccf8b99a10e923437495ff74b7b6cec5733079ed3d5565ff7bc: Status 404 returned error can't find the container with id c4b9e13aa4addccf8b99a10e923437495ff74b7b6cec5733079ed3d5565ff7bc Jul 13 22:08:58.111283 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.111257 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz"] Jul 13 22:08:58.637209 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.637174 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" event={"ID":"5ad98b2f-3bf7-4019-9076-361c089c95b5","Type":"ContainerStarted","Data":"c4b9e13aa4addccf8b99a10e923437495ff74b7b6cec5733079ed3d5565ff7bc"} Jul 13 22:08:58.689980 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.689951 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:58.893011 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.892930 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:08:58.900153 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.900131 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7"] Jul 13 22:08:58.903546 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.903530 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" Jul 13 22:08:58.906110 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.906074 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 13 22:08:58.906255 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.906239 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wvrcz\"" Jul 13 22:08:58.926755 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.926734 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7"] Jul 13 22:08:58.948789 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.948763 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-5czcx"] Jul 13 22:08:58.952521 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.952504 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" Jul 13 22:08:58.968480 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.968457 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-5czcx"] Jul 13 22:08:58.990285 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.990060 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwkx7\" (UniqueName: \"kubernetes.io/projected/05c0b422-638c-4b6f-a42e-c75fe345e9d7-kube-api-access-rwkx7\") pod \"payload-pre-processing-74584bb9f8-8q5b7\" (UID: \"05c0b422-638c-4b6f-a42e-c75fe345e9d7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" Jul 13 22:08:58.990285 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:58.990100 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05c0b422-638c-4b6f-a42e-c75fe345e9d7-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-8q5b7\" (UID: \"05c0b422-638c-4b6f-a42e-c75fe345e9d7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" Jul 13 22:08:59.091444 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.091408 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0e46651-ae37-4f23-862d-a628630d6e40-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-5czcx\" (UID: \"a0e46651-ae37-4f23-862d-a628630d6e40\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" Jul 13 22:08:59.091620 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.091463 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9vsd\" (UniqueName: \"kubernetes.io/projected/a0e46651-ae37-4f23-862d-a628630d6e40-kube-api-access-v9vsd\") pod \"payload-processing-f9b8d9b9b-5czcx\" (UID: \"a0e46651-ae37-4f23-862d-a628630d6e40\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" Jul 13 22:08:59.091620 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.091549 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwkx7\" (UniqueName: \"kubernetes.io/projected/05c0b422-638c-4b6f-a42e-c75fe345e9d7-kube-api-access-rwkx7\") pod \"payload-pre-processing-74584bb9f8-8q5b7\" (UID: \"05c0b422-638c-4b6f-a42e-c75fe345e9d7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" Jul 13 22:08:59.091620 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.091575 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05c0b422-638c-4b6f-a42e-c75fe345e9d7-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-8q5b7\" (UID: \"05c0b422-638c-4b6f-a42e-c75fe345e9d7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" Jul 13 22:08:59.092436 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.092404 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05c0b422-638c-4b6f-a42e-c75fe345e9d7-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-8q5b7\" (UID: \"05c0b422-638c-4b6f-a42e-c75fe345e9d7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" Jul 13 22:08:59.117439 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.117411 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwkx7\" (UniqueName: \"kubernetes.io/projected/05c0b422-638c-4b6f-a42e-c75fe345e9d7-kube-api-access-rwkx7\") pod \"payload-pre-processing-74584bb9f8-8q5b7\" (UID: \"05c0b422-638c-4b6f-a42e-c75fe345e9d7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" Jul 13 22:08:59.192795 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.192721 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0e46651-ae37-4f23-862d-a628630d6e40-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-5czcx\" (UID: \"a0e46651-ae37-4f23-862d-a628630d6e40\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" Jul 13 22:08:59.192795 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.192780 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9vsd\" (UniqueName: \"kubernetes.io/projected/a0e46651-ae37-4f23-862d-a628630d6e40-kube-api-access-v9vsd\") pod \"payload-processing-f9b8d9b9b-5czcx\" (UID: \"a0e46651-ae37-4f23-862d-a628630d6e40\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" Jul 13 22:08:59.193385 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.193366 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0e46651-ae37-4f23-862d-a628630d6e40-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-5czcx\" (UID: \"a0e46651-ae37-4f23-862d-a628630d6e40\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" Jul 13 22:08:59.214073 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.214050 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" Jul 13 22:08:59.216509 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.216490 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9vsd\" (UniqueName: \"kubernetes.io/projected/a0e46651-ae37-4f23-862d-a628630d6e40-kube-api-access-v9vsd\") pod \"payload-processing-f9b8d9b9b-5czcx\" (UID: \"a0e46651-ae37-4f23-862d-a628630d6e40\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" Jul 13 22:08:59.263867 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.263823 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" Jul 13 22:08:59.376653 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.376615 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7"] Jul 13 22:08:59.381375 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:08:59.381345 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05c0b422_638c_4b6f_a42e_c75fe345e9d7.slice/crio-1c17c3b7192082b7c7e3f86573de5dc6c90e35388fa3cebbf0be64df03340263 WatchSource:0}: Error finding container 1c17c3b7192082b7c7e3f86573de5dc6c90e35388fa3cebbf0be64df03340263: Status 404 returned error can't find the container with id 1c17c3b7192082b7c7e3f86573de5dc6c90e35388fa3cebbf0be64df03340263 Jul 13 22:08:59.422938 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.422914 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-5czcx"] Jul 13 22:08:59.425161 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:08:59.425134 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e46651_ae37_4f23_862d_a628630d6e40.slice/crio-eeef97617cec423ba47dfe63b4da5fc07c9b1604b17cba6da6a1e2500553aadd WatchSource:0}: Error finding container eeef97617cec423ba47dfe63b4da5fc07c9b1604b17cba6da6a1e2500553aadd: Status 404 returned error can't find the container with id eeef97617cec423ba47dfe63b4da5fc07c9b1604b17cba6da6a1e2500553aadd Jul 13 22:08:59.643046 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.643011 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" event={"ID":"a0e46651-ae37-4f23-862d-a628630d6e40","Type":"ContainerStarted","Data":"eeef97617cec423ba47dfe63b4da5fc07c9b1604b17cba6da6a1e2500553aadd"} Jul 13 22:08:59.644255 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:08:59.644220 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" event={"ID":"05c0b422-638c-4b6f-a42e-c75fe345e9d7","Type":"ContainerStarted","Data":"1c17c3b7192082b7c7e3f86573de5dc6c90e35388fa3cebbf0be64df03340263"} Jul 13 22:09:00.651180 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:00.651145 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" event={"ID":"5ad98b2f-3bf7-4019-9076-361c089c95b5","Type":"ContainerStarted","Data":"9d4c0494a943308ba2af07fbbba55ccfa78505ca2dc47f18eab271300bac07d4"} Jul 13 22:09:00.651583 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:00.651270 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:09:01.656594 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:01.656561 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" event={"ID":"a0e46651-ae37-4f23-862d-a628630d6e40","Type":"ContainerStarted","Data":"32c521d5a33e0f7ac5fcd1f0bc0e1dcabd9fe81aff4703920e8fb07eeb4e3e62"} Jul 13 22:09:01.657081 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:01.656653 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" Jul 13 22:09:01.658120 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:01.658098 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" event={"ID":"05c0b422-638c-4b6f-a42e-c75fe345e9d7","Type":"ContainerStarted","Data":"be08cf26abdc5358c8bede327f9878a9da31d313cdec47aee8da94fda437325e"} Jul 13 22:09:01.658208 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:01.658155 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" Jul 13 22:09:01.683937 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:01.683897 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" podStartSLOduration=2.385369421 podStartE2EDuration="3.683880267s" totalImagesPullingTime="1.298510846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:09:01.682321269 +0000 UTC m=+550.865165518" watchObservedRunningTime="2026-07-13 22:09:01.683880267 +0000 UTC m=+550.866724502" Jul 13 22:09:01.685233 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:01.685197 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" podStartSLOduration=2.920713 podStartE2EDuration="4.685186747s" totalImagesPullingTime="1.764473747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:09:00.685630916 +0000 UTC m=+549.868475146" watchObservedRunningTime="2026-07-13 22:09:01.685186747 +0000 UTC m=+550.868030978" Jul 13 22:09:01.720949 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:01.720913 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" podStartSLOduration=2.381308738 podStartE2EDuration="3.720900224s" totalImagesPullingTime="1.339591486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:09:01.719940375 +0000 UTC m=+550.902784634" watchObservedRunningTime="2026-07-13 22:09:01.720900224 +0000 UTC m=+550.903744454" Jul 13 22:09:06.663944 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:06.663918 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6454d8c544-jcfgz" Jul 13 22:09:08.152796 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:08.152761 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:09:08.247438 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:08.247404 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:09:12.664740 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:12.664710 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-5czcx" Jul 13 22:09:12.665237 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:12.664793 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8q5b7" Jul 13 22:09:13.315590 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:13.315555 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:09:19.504137 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:19.504105 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:09:35.572907 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.572872 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4"] Jul 13 22:09:35.576327 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.576305 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.579781 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.579760 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 22:09:35.580131 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.580113 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 13 22:09:35.580248 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.580133 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 22:09:35.580248 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.580134 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gfrsn\"" Jul 13 22:09:35.590462 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.590443 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4"] Jul 13 22:09:35.597790 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.597770 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.597907 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.597807 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.597907 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.597874 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbp5g\" (UniqueName: \"kubernetes.io/projected/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-kube-api-access-rbp5g\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.597907 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.597900 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.598066 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.597939 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.598066 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.598014 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.621696 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.621664 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:09:35.698578 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.698551 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.698705 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.698590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.698705 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.698631 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbp5g\" (UniqueName: \"kubernetes.io/projected/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-kube-api-access-rbp5g\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.698705 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.698651 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.698705 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.698692 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.698947 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.698730 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.699030 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.699001 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.699030 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.699023 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.699212 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.699111 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.701030 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.701010 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.701225 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.701208 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.708871 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.708836 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbp5g\" (UniqueName: \"kubernetes.io/projected/b5a56eb6-d495-4d64-a600-7cd96c8b62c9-kube-api-access-rbp5g\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vnbd4\" (UID: \"b5a56eb6-d495-4d64-a600-7cd96c8b62c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:35.886051 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:35.886019 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:36.009231 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:36.009209 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4"] Jul 13 22:09:36.011389 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:09:36.011357 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a56eb6_d495_4d64_a600_7cd96c8b62c9.slice/crio-813503a00576aade4dc3941da063ae4e7800bbbc689875df415343a96a65836b WatchSource:0}: Error finding container 813503a00576aade4dc3941da063ae4e7800bbbc689875df415343a96a65836b: Status 404 returned error can't find the container with id 813503a00576aade4dc3941da063ae4e7800bbbc689875df415343a96a65836b Jul 13 22:09:36.816622 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:36.816584 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" event={"ID":"b5a56eb6-d495-4d64-a600-7cd96c8b62c9","Type":"ContainerStarted","Data":"813503a00576aade4dc3941da063ae4e7800bbbc689875df415343a96a65836b"} Jul 13 22:09:37.822253 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:37.822210 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" event={"ID":"b5a56eb6-d495-4d64-a600-7cd96c8b62c9","Type":"ContainerStarted","Data":"3c7acc872a0978d81ce1ca7293dfeca220b5af19f980a6e27b5004d1e31136b6"} Jul 13 22:09:37.841730 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:37.841675 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" podStartSLOduration=1.842606672 podStartE2EDuration="2.841656513s" totalImagesPullingTime="999.049841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:09:37.838831805 +0000 UTC m=+587.021676056" watchObservedRunningTime="2026-07-13 22:09:37.841656513 +0000 UTC m=+587.024500743" Jul 13 22:09:44.092628 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.092588 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps"] Jul 13 22:09:44.095036 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.095015 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.099650 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.099628 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 13 22:09:44.117703 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.117678 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps"] Jul 13 22:09:44.165980 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.165951 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.166109 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.166000 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.166109 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.166028 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/539748de-2362-4ece-834a-47d4af878dec-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.166109 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.166076 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x72gm\" (UniqueName: \"kubernetes.io/projected/539748de-2362-4ece-834a-47d4af878dec-kube-api-access-x72gm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.166208 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.166118 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.166208 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.166136 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.188755 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.188720 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:09:44.266663 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.266637 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.266663 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.266666 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.266828 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.266707 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.266828 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.266737 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.266828 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.266754 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/539748de-2362-4ece-834a-47d4af878dec-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.267011 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.266890 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x72gm\" (UniqueName: \"kubernetes.io/projected/539748de-2362-4ece-834a-47d4af878dec-kube-api-access-x72gm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.267049 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.267035 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.267170 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.267148 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.267283 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.267268 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.268921 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.268900 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/539748de-2362-4ece-834a-47d4af878dec-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.269104 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.269088 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/539748de-2362-4ece-834a-47d4af878dec-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.276707 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.276687 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x72gm\" (UniqueName: \"kubernetes.io/projected/539748de-2362-4ece-834a-47d4af878dec-kube-api-access-x72gm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps\" (UID: \"539748de-2362-4ece-834a-47d4af878dec\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.404143 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.404115 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:44.535792 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.535765 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps"] Jul 13 22:09:44.537368 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:09:44.537344 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod539748de_2362_4ece_834a_47d4af878dec.slice/crio-9afff9d510620e7443a9a25062bc46e615c4615219f5e06d2fddf8c3923f01aa WatchSource:0}: Error finding container 9afff9d510620e7443a9a25062bc46e615c4615219f5e06d2fddf8c3923f01aa: Status 404 returned error can't find the container with id 9afff9d510620e7443a9a25062bc46e615c4615219f5e06d2fddf8c3923f01aa Jul 13 22:09:44.854473 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.854441 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" event={"ID":"539748de-2362-4ece-834a-47d4af878dec","Type":"ContainerStarted","Data":"6285025b1b61e023f2095fd7861403351356974dba1576d8332b2240558bc21b"} Jul 13 22:09:44.854627 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.854500 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" event={"ID":"539748de-2362-4ece-834a-47d4af878dec","Type":"ContainerStarted","Data":"9afff9d510620e7443a9a25062bc46e615c4615219f5e06d2fddf8c3923f01aa"} Jul 13 22:09:44.878992 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:44.878938 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" podStartSLOduration=0.64403822 podStartE2EDuration="878.91882ms" totalImagesPullingTime="234.8806ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:09:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:09:44.877153004 +0000 UTC m=+594.059997252" watchObservedRunningTime="2026-07-13 22:09:44.87891882 +0000 UTC m=+594.061763050" Jul 13 22:09:45.886154 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:45.886125 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:45.886506 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:45.886207 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:45.898138 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:45.898115 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:46.874629 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:46.874603 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vnbd4" Jul 13 22:09:51.215818 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:51.215791 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:09:51.216264 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:51.216159 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:09:54.404700 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:54.404657 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:54.406067 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:54.404717 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:54.417092 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:54.417070 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:54.910962 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:54.910932 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps" Jul 13 22:09:57.697430 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:09:57.697395 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:10:16.685712 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:10:16.685625 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:10:27.587493 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:10:27.587460 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:11:07.887618 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:11:07.887558 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:11:14.371549 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:11:14.371503 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:11:14.471292 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:11:14.471263 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:11:15.514733 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:11:15.514698 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:11:15.618868 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:11:15.618809 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:11:18.793180 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:11:18.793144 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:11:18.854557 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:11:18.854527 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:11:37.911754 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:11:37.911715 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:11:38.012010 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:11:38.011978 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:12:19.978087 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:19.978050 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:12:20.034454 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:20.034416 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:12:25.805864 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:25.805819 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:12:25.848911 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:25.848882 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6dc46d8554-p2sgf"] Jul 13 22:12:25.849145 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:25.849123 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" podUID="d8d8470f-478f-4a36-8b90-de6504a8a76a" containerName="manager" containerID="cri-o://58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078" gracePeriod=30 Jul 13 22:12:26.090355 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.090330 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:12:26.179980 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.179949 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w4qm\" (UniqueName: \"kubernetes.io/projected/d8d8470f-478f-4a36-8b90-de6504a8a76a-kube-api-access-9w4qm\") pod \"d8d8470f-478f-4a36-8b90-de6504a8a76a\" (UID: \"d8d8470f-478f-4a36-8b90-de6504a8a76a\") " Jul 13 22:12:26.179980 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.179981 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8d8470f-478f-4a36-8b90-de6504a8a76a-webhook-cert\") pod \"d8d8470f-478f-4a36-8b90-de6504a8a76a\" (UID: \"d8d8470f-478f-4a36-8b90-de6504a8a76a\") " Jul 13 22:12:26.182001 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.181973 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8d8470f-478f-4a36-8b90-de6504a8a76a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d8d8470f-478f-4a36-8b90-de6504a8a76a" (UID: "d8d8470f-478f-4a36-8b90-de6504a8a76a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:12:26.182001 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.181980 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8d8470f-478f-4a36-8b90-de6504a8a76a-kube-api-access-9w4qm" (OuterVolumeSpecName: "kube-api-access-9w4qm") pod "d8d8470f-478f-4a36-8b90-de6504a8a76a" (UID: "d8d8470f-478f-4a36-8b90-de6504a8a76a"). InnerVolumeSpecName "kube-api-access-9w4qm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:12:26.281016 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.280987 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w4qm\" (UniqueName: \"kubernetes.io/projected/d8d8470f-478f-4a36-8b90-de6504a8a76a-kube-api-access-9w4qm\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:12:26.281016 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.281011 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8d8470f-478f-4a36-8b90-de6504a8a76a-webhook-cert\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:12:26.580981 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.580949 2570 generic.go:356] "Generic (PLEG): container finished" podID="d8d8470f-478f-4a36-8b90-de6504a8a76a" containerID="58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078" exitCode=0 Jul 13 22:12:26.581141 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.580985 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" Jul 13 22:12:26.581141 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.581005 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" event={"ID":"d8d8470f-478f-4a36-8b90-de6504a8a76a","Type":"ContainerDied","Data":"58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078"} Jul 13 22:12:26.581141 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.581055 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dc46d8554-p2sgf" event={"ID":"d8d8470f-478f-4a36-8b90-de6504a8a76a","Type":"ContainerDied","Data":"12feb823812595195210e9577ad28f250441bc3d410faa6ae450b6c42b30edf0"} Jul 13 22:12:26.581141 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.581078 2570 scope.go:117] "RemoveContainer" containerID="58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078" Jul 13 22:12:26.589918 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.589898 2570 scope.go:117] "RemoveContainer" containerID="58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078" Jul 13 22:12:26.590152 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:12:26.590132 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078\": container with ID starting with 58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078 not found: ID does not exist" containerID="58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078" Jul 13 22:12:26.590199 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.590159 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078"} err="failed to get container status \"58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078\": rpc error: code = NotFound desc = could not find container \"58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078\": container with ID starting with 58550d76d987a0089849b1da6819698ecfc17cf9e97c54cf00fe709decd76078 not found: ID does not exist" Jul 13 22:12:26.600883 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.600845 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6dc46d8554-p2sgf"] Jul 13 22:12:26.604415 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:26.604395 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6dc46d8554-p2sgf"] Jul 13 22:12:27.227429 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.227393 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:12:27.316440 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.316409 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8d8470f-478f-4a36-8b90-de6504a8a76a" path="/var/lib/kubelet/pods/d8d8470f-478f-4a36-8b90-de6504a8a76a/volumes" Jul 13 22:12:27.332841 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.332812 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6dc46d8554-mq6tm"] Jul 13 22:12:27.333176 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.333164 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8d8470f-478f-4a36-8b90-de6504a8a76a" containerName="manager" Jul 13 22:12:27.333219 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.333178 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8d8470f-478f-4a36-8b90-de6504a8a76a" containerName="manager" Jul 13 22:12:27.333260 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.333250 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8d8470f-478f-4a36-8b90-de6504a8a76a" containerName="manager" Jul 13 22:12:27.337538 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.337522 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dc46d8554-mq6tm" Jul 13 22:12:27.339894 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.339874 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 22:12:27.340046 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.339876 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d99t2\"" Jul 13 22:12:27.340046 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.339875 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 22:12:27.347542 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.347524 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dc46d8554-mq6tm"] Jul 13 22:12:27.493249 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.493176 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3944875c-dd62-4f5e-9e75-c3cf8dbe69bc-webhook-cert\") pod \"maas-controller-6dc46d8554-mq6tm\" (UID: \"3944875c-dd62-4f5e-9e75-c3cf8dbe69bc\") " pod="opendatahub/maas-controller-6dc46d8554-mq6tm" Jul 13 22:12:27.493249 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.493207 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvzfv\" (UniqueName: \"kubernetes.io/projected/3944875c-dd62-4f5e-9e75-c3cf8dbe69bc-kube-api-access-wvzfv\") pod \"maas-controller-6dc46d8554-mq6tm\" (UID: \"3944875c-dd62-4f5e-9e75-c3cf8dbe69bc\") " pod="opendatahub/maas-controller-6dc46d8554-mq6tm" Jul 13 22:12:27.594603 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.594576 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3944875c-dd62-4f5e-9e75-c3cf8dbe69bc-webhook-cert\") pod \"maas-controller-6dc46d8554-mq6tm\" (UID: \"3944875c-dd62-4f5e-9e75-c3cf8dbe69bc\") " pod="opendatahub/maas-controller-6dc46d8554-mq6tm" Jul 13 22:12:27.594603 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.594606 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvzfv\" (UniqueName: \"kubernetes.io/projected/3944875c-dd62-4f5e-9e75-c3cf8dbe69bc-kube-api-access-wvzfv\") pod \"maas-controller-6dc46d8554-mq6tm\" (UID: \"3944875c-dd62-4f5e-9e75-c3cf8dbe69bc\") " pod="opendatahub/maas-controller-6dc46d8554-mq6tm" Jul 13 22:12:27.596974 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.596954 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3944875c-dd62-4f5e-9e75-c3cf8dbe69bc-webhook-cert\") pod \"maas-controller-6dc46d8554-mq6tm\" (UID: \"3944875c-dd62-4f5e-9e75-c3cf8dbe69bc\") " pod="opendatahub/maas-controller-6dc46d8554-mq6tm" Jul 13 22:12:27.603120 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.603098 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvzfv\" (UniqueName: \"kubernetes.io/projected/3944875c-dd62-4f5e-9e75-c3cf8dbe69bc-kube-api-access-wvzfv\") pod \"maas-controller-6dc46d8554-mq6tm\" (UID: \"3944875c-dd62-4f5e-9e75-c3cf8dbe69bc\") " pod="opendatahub/maas-controller-6dc46d8554-mq6tm" Jul 13 22:12:27.648782 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.648762 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dc46d8554-mq6tm" Jul 13 22:12:27.775865 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.775825 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dc46d8554-mq6tm"] Jul 13 22:12:27.777910 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:12:27.777882 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3944875c_dd62_4f5e_9e75_c3cf8dbe69bc.slice/crio-a0bbaf5535a5a86b5ad8226977f5b53353371cdbfae721fee44bffdc4e5a2f88 WatchSource:0}: Error finding container a0bbaf5535a5a86b5ad8226977f5b53353371cdbfae721fee44bffdc4e5a2f88: Status 404 returned error can't find the container with id a0bbaf5535a5a86b5ad8226977f5b53353371cdbfae721fee44bffdc4e5a2f88 Jul 13 22:12:27.779370 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:27.779347 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 22:12:28.592054 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:28.591962 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dc46d8554-mq6tm" event={"ID":"3944875c-dd62-4f5e-9e75-c3cf8dbe69bc","Type":"ContainerStarted","Data":"217021c96e04fd00773df58a9b1929fbc292c1ea8721410d729e9ba0541e95f9"} Jul 13 22:12:28.592054 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:28.592019 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dc46d8554-mq6tm" event={"ID":"3944875c-dd62-4f5e-9e75-c3cf8dbe69bc","Type":"ContainerStarted","Data":"a0bbaf5535a5a86b5ad8226977f5b53353371cdbfae721fee44bffdc4e5a2f88"} Jul 13 22:12:28.592530 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:28.592082 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6dc46d8554-mq6tm" Jul 13 22:12:28.608805 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:28.608756 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6dc46d8554-mq6tm" podStartSLOduration=1.107910238 podStartE2EDuration="1.608738275s" totalImagesPullingTime="500.828037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:12:28.608053508 +0000 UTC m=+757.790897752" watchObservedRunningTime="2026-07-13 22:12:28.608738275 +0000 UTC m=+757.791582514" Jul 13 22:12:39.598798 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:12:39.598763 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6dc46d8554-mq6tm" Jul 13 22:13:00.835565 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:13:00.835515 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:13:00.931457 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:13:00.931417 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:13:01.061780 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:13:01.061749 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:13:06.864459 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:13:06.864419 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:13:06.970806 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:13:06.970772 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:13:07.064457 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:13:07.064422 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:13:07.230308 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:13:07.230218 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:13:13.073267 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:13:13.073171 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:13:13.174004 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:13:13.173959 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:13:54.972113 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:13:54.972076 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:14:51.241898 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:14:51.241868 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:14:51.244528 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:14:51.244503 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:16:17.649734 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:16:17.649654 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:16:18.825891 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:16:18.825861 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:18:58.548290 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:18:58.548257 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:19:51.266947 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:19:51.266919 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:19:51.270644 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:19:51.270623 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:21:33.360089 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:21:33.360059 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:21:34.510233 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:21:34.510204 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:23:23.283719 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:23:23.283683 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:23:52.327520 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:23:52.327433 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:23:59.728036 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:23:59.727988 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:07.973802 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:07.973768 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:13.042827 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:13.042789 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:15.978157 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:15.978121 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:16.069253 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:16.069218 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:16.412831 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:16.412798 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:22.875204 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:22.875176 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:22.984784 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:22.984752 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:23.132756 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:23.132680 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:23.214941 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:23.214914 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:24.296231 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.296201 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:24.391000 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.390974 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:24.553406 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.553330 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l"] Jul 13 22:24:24.556525 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.556506 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:24.558744 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.558724 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d7fae36e-serving-cert\"" Jul 13 22:24:24.631056 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.631027 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l"] Jul 13 22:24:24.640250 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.640223 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:24.643375 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.643349 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-maas-api-tls\") pod \"maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:24.643482 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.643390 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk2jc\" (UniqueName: \"kubernetes.io/projected/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-kube-api-access-mk2jc\") pod \"maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:24.643482 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.643419 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-openshift-service-ca\") pod \"maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:24.744605 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.744570 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-maas-api-tls\") pod \"maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:24.744605 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.744605 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk2jc\" (UniqueName: \"kubernetes.io/projected/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-kube-api-access-mk2jc\") pod \"maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:24.744811 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.744628 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-openshift-service-ca\") pod \"maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:24.744811 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:24:24.744717 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-serving-cert: secret "maas-api-e2e-ait-d7fae36e-serving-cert" not found Jul 13 22:24:24.744811 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:24:24.744794 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-maas-api-tls podName:8dec33b2-f9f1-4841-9e3e-71827d1a4cd2 nodeName:}" failed. No retries permitted until 2026-07-13 22:24:25.24477854 +0000 UTC m=+1474.427622753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-maas-api-tls") pod "maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" (UID: "8dec33b2-f9f1-4841-9e3e-71827d1a4cd2") : secret "maas-api-e2e-ait-d7fae36e-serving-cert" not found Jul 13 22:24:24.745316 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.745299 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-openshift-service-ca\") pod \"maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:24.759819 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:24.759798 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk2jc\" (UniqueName: \"kubernetes.io/projected/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-kube-api-access-mk2jc\") pod \"maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:25.030224 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:25.030194 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:25.202184 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:25.202149 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:25.247970 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:25.247935 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-maas-api-tls\") pod \"maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:25.250332 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:25.250303 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-maas-api-tls\") pod \"maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:25.307395 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:25.307335 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:25.466567 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:25.466537 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:25.600977 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:25.600955 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l"] Jul 13 22:24:25.603532 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:24:25.603506 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dec33b2_f9f1_4841_9e3e_71827d1a4cd2.slice/crio-3c21e18f98b06c7f4349d4a4d6d9844792137a1cedc5756055bba78cbca8c56b WatchSource:0}: Error finding container 3c21e18f98b06c7f4349d4a4d6d9844792137a1cedc5756055bba78cbca8c56b: Status 404 returned error can't find the container with id 3c21e18f98b06c7f4349d4a4d6d9844792137a1cedc5756055bba78cbca8c56b Jul 13 22:24:25.605084 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:25.605065 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 22:24:25.811562 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:25.811531 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" event={"ID":"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2","Type":"ContainerStarted","Data":"3c21e18f98b06c7f4349d4a4d6d9844792137a1cedc5756055bba78cbca8c56b"} Jul 13 22:24:26.818637 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:26.818592 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" event={"ID":"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2","Type":"ContainerStarted","Data":"cd53b9f7624af184c368d23582f2d2ce9b349ec640345ac4023013eeeca1352f"} Jul 13 22:24:26.819000 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:26.818713 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:26.827668 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:26.827645 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:26.835969 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:26.835931 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" podStartSLOduration=2.210861626 podStartE2EDuration="2.835916303s" totalImagesPullingTime="625.054677ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:24:26.833988956 +0000 UTC m=+1476.016833212" watchObservedRunningTime="2026-07-13 22:24:26.835916303 +0000 UTC m=+1476.018760553" Jul 13 22:24:32.829367 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:32.829339 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:24:38.610569 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:38.610534 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:24:51.291958 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:51.291925 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:24:51.297829 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:24:51.297810 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:25:29.903136 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:29.903061 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l"] Jul 13 22:25:29.903713 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:29.903265 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" podUID="8dec33b2-f9f1-4841-9e3e-71827d1a4cd2" containerName="maas-api" containerID="cri-o://cd53b9f7624af184c368d23582f2d2ce9b349ec640345ac4023013eeeca1352f" gracePeriod=30 Jul 13 22:25:29.935653 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:29.935627 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:30.109708 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.109679 2570 generic.go:356] "Generic (PLEG): container finished" podID="8dec33b2-f9f1-4841-9e3e-71827d1a4cd2" containerID="cd53b9f7624af184c368d23582f2d2ce9b349ec640345ac4023013eeeca1352f" exitCode=0 Jul 13 22:25:30.109894 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.109758 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" event={"ID":"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2","Type":"ContainerDied","Data":"cd53b9f7624af184c368d23582f2d2ce9b349ec640345ac4023013eeeca1352f"} Jul 13 22:25:30.141619 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.141591 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:25:30.274740 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.274672 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-maas-api-tls\") pod \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " Jul 13 22:25:30.274740 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.274702 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mk2jc\" (UniqueName: \"kubernetes.io/projected/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-kube-api-access-mk2jc\") pod \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " Jul 13 22:25:30.274940 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.274748 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-openshift-service-ca\") pod \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\" (UID: \"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2\") " Jul 13 22:25:30.275155 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.275133 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8dec33b2-f9f1-4841-9e3e-71827d1a4cd2" (UID: "8dec33b2-f9f1-4841-9e3e-71827d1a4cd2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:25:30.276841 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.276811 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8dec33b2-f9f1-4841-9e3e-71827d1a4cd2" (UID: "8dec33b2-f9f1-4841-9e3e-71827d1a4cd2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:25:30.276841 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.276817 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-kube-api-access-mk2jc" (OuterVolumeSpecName: "kube-api-access-mk2jc") pod "8dec33b2-f9f1-4841-9e3e-71827d1a4cd2" (UID: "8dec33b2-f9f1-4841-9e3e-71827d1a4cd2"). InnerVolumeSpecName "kube-api-access-mk2jc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:25:30.375742 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.375716 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:25:30.375742 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.375740 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk2jc\" (UniqueName: \"kubernetes.io/projected/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-kube-api-access-mk2jc\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:25:30.375905 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:30.375750 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:25:31.115086 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:31.115061 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" Jul 13 22:25:31.115478 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:31.115086 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l" event={"ID":"8dec33b2-f9f1-4841-9e3e-71827d1a4cd2","Type":"ContainerDied","Data":"3c21e18f98b06c7f4349d4a4d6d9844792137a1cedc5756055bba78cbca8c56b"} Jul 13 22:25:31.115478 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:31.115137 2570 scope.go:117] "RemoveContainer" containerID="cd53b9f7624af184c368d23582f2d2ce9b349ec640345ac4023013eeeca1352f" Jul 13 22:25:31.137564 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:31.137538 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l"] Jul 13 22:25:31.142425 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:31.142405 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d7fae36e-7688cffc9d-dwj5l"] Jul 13 22:25:31.316303 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:31.316274 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dec33b2-f9f1-4841-9e3e-71827d1a4cd2" path="/var/lib/kubelet/pods/8dec33b2-f9f1-4841-9e3e-71827d1a4cd2/volumes" Jul 13 22:25:34.400875 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:34.400822 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:40.536063 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:40.536024 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:40.632203 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:40.632171 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:44.645048 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:44.645014 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:45.001707 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:45.001623 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:45.109073 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:45.109040 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:45.280235 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:45.280168 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:46.536298 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.536270 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:46.700755 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.700721 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb"] Jul 13 22:25:46.701464 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.701440 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dec33b2-f9f1-4841-9e3e-71827d1a4cd2" containerName="maas-api" Jul 13 22:25:46.701583 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.701501 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dec33b2-f9f1-4841-9e3e-71827d1a4cd2" containerName="maas-api" Jul 13 22:25:46.701646 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.701627 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dec33b2-f9f1-4841-9e3e-71827d1a4cd2" containerName="maas-api" Jul 13 22:25:46.705896 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.705876 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:46.707861 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.707826 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-65448f9f-serving-cert\"" Jul 13 22:25:46.716071 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.716049 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb"] Jul 13 22:25:46.739929 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.739903 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:46.821147 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.821070 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad9adcb4-d3cf-48de-9f91-aace8fac975a-openshift-service-ca\") pod \"maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:46.821147 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.821116 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad9adcb4-d3cf-48de-9f91-aace8fac975a-maas-api-tls\") pod \"maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:46.821312 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.821216 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cg8q\" (UniqueName: \"kubernetes.io/projected/ad9adcb4-d3cf-48de-9f91-aace8fac975a-kube-api-access-5cg8q\") pod \"maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:46.866800 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.866767 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:46.922064 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.922040 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cg8q\" (UniqueName: \"kubernetes.io/projected/ad9adcb4-d3cf-48de-9f91-aace8fac975a-kube-api-access-5cg8q\") pod \"maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:46.922199 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.922092 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad9adcb4-d3cf-48de-9f91-aace8fac975a-openshift-service-ca\") pod \"maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:46.922199 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.922123 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad9adcb4-d3cf-48de-9f91-aace8fac975a-maas-api-tls\") pod \"maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:46.922667 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.922644 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad9adcb4-d3cf-48de-9f91-aace8fac975a-openshift-service-ca\") pod \"maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:46.924456 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.924437 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad9adcb4-d3cf-48de-9f91-aace8fac975a-maas-api-tls\") pod \"maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:46.929738 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:46.929714 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cg8q\" (UniqueName: \"kubernetes.io/projected/ad9adcb4-d3cf-48de-9f91-aace8fac975a-kube-api-access-5cg8q\") pod \"maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:47.019781 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:47.019756 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:47.170797 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:47.170772 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb"] Jul 13 22:25:47.173570 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:25:47.173543 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9adcb4_d3cf_48de_9f91_aace8fac975a.slice/crio-e0b865d373ad43cc3db8c5e3480fa74b890fe96f052d02de81d2bae2c3548a4d WatchSource:0}: Error finding container e0b865d373ad43cc3db8c5e3480fa74b890fe96f052d02de81d2bae2c3548a4d: Status 404 returned error can't find the container with id e0b865d373ad43cc3db8c5e3480fa74b890fe96f052d02de81d2bae2c3548a4d Jul 13 22:25:47.191501 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:47.191472 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" event={"ID":"ad9adcb4-d3cf-48de-9f91-aace8fac975a","Type":"ContainerStarted","Data":"e0b865d373ad43cc3db8c5e3480fa74b890fe96f052d02de81d2bae2c3548a4d"} Jul 13 22:25:47.403318 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:47.403281 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:47.506073 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:47.506043 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:48.197690 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:48.197651 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" event={"ID":"ad9adcb4-d3cf-48de-9f91-aace8fac975a","Type":"ContainerStarted","Data":"6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351"} Jul 13 22:25:48.198162 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:48.197787 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:48.219933 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:48.219886 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" podStartSLOduration=1.6787621050000001 podStartE2EDuration="2.219871987s" totalImagesPullingTime="541.109882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:25:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:25:48.218279893 +0000 UTC m=+1557.401124125" watchObservedRunningTime="2026-07-13 22:25:48.219871987 +0000 UTC m=+1557.402716218" Jul 13 22:25:51.125093 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:51.125067 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:51.225087 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:51.225031 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:25:54.208094 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:54.208065 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:25:54.347804 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:25:54.347771 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:37.810715 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:37.810675 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:51.093015 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.092936 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:51.184901 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.184864 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:51.608822 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.608786 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb"] Jul 13 22:26:51.609096 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.609065 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" podUID="ad9adcb4-d3cf-48de-9f91-aace8fac975a" containerName="maas-api" containerID="cri-o://6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351" gracePeriod=30 Jul 13 22:26:51.648576 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.648544 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:51.857179 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.857156 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:26:51.880094 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.880023 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cg8q\" (UniqueName: \"kubernetes.io/projected/ad9adcb4-d3cf-48de-9f91-aace8fac975a-kube-api-access-5cg8q\") pod \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " Jul 13 22:26:51.880094 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.880076 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad9adcb4-d3cf-48de-9f91-aace8fac975a-openshift-service-ca\") pod \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " Jul 13 22:26:51.880302 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.880107 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad9adcb4-d3cf-48de-9f91-aace8fac975a-maas-api-tls\") pod \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\" (UID: \"ad9adcb4-d3cf-48de-9f91-aace8fac975a\") " Jul 13 22:26:51.880494 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.880464 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad9adcb4-d3cf-48de-9f91-aace8fac975a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ad9adcb4-d3cf-48de-9f91-aace8fac975a" (UID: "ad9adcb4-d3cf-48de-9f91-aace8fac975a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:26:51.882216 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.882185 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9adcb4-d3cf-48de-9f91-aace8fac975a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ad9adcb4-d3cf-48de-9f91-aace8fac975a" (UID: "ad9adcb4-d3cf-48de-9f91-aace8fac975a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:26:51.882321 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.882255 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad9adcb4-d3cf-48de-9f91-aace8fac975a-kube-api-access-5cg8q" (OuterVolumeSpecName: "kube-api-access-5cg8q") pod "ad9adcb4-d3cf-48de-9f91-aace8fac975a" (UID: "ad9adcb4-d3cf-48de-9f91-aace8fac975a"). InnerVolumeSpecName "kube-api-access-5cg8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:26:51.981716 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.981677 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cg8q\" (UniqueName: \"kubernetes.io/projected/ad9adcb4-d3cf-48de-9f91-aace8fac975a-kube-api-access-5cg8q\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:26:51.981716 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.981710 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad9adcb4-d3cf-48de-9f91-aace8fac975a-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:26:51.981716 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:51.981723 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad9adcb4-d3cf-48de-9f91-aace8fac975a-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:26:52.496359 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:52.496330 2570 generic.go:356] "Generic (PLEG): container finished" podID="ad9adcb4-d3cf-48de-9f91-aace8fac975a" containerID="6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351" exitCode=0 Jul 13 22:26:52.496839 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:52.496379 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" event={"ID":"ad9adcb4-d3cf-48de-9f91-aace8fac975a","Type":"ContainerDied","Data":"6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351"} Jul 13 22:26:52.496839 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:52.496394 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" Jul 13 22:26:52.496839 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:52.496420 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb" event={"ID":"ad9adcb4-d3cf-48de-9f91-aace8fac975a","Type":"ContainerDied","Data":"e0b865d373ad43cc3db8c5e3480fa74b890fe96f052d02de81d2bae2c3548a4d"} Jul 13 22:26:52.496839 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:52.496441 2570 scope.go:117] "RemoveContainer" containerID="6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351" Jul 13 22:26:52.505023 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:52.505006 2570 scope.go:117] "RemoveContainer" containerID="6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351" Jul 13 22:26:52.505281 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:26:52.505263 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351\": container with ID starting with 6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351 not found: ID does not exist" containerID="6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351" Jul 13 22:26:52.505340 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:52.505289 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351"} err="failed to get container status \"6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351\": rpc error: code = NotFound desc = could not find container \"6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351\": container with ID starting with 6a3dca661efd69c8d8a2d02dd5a48644e417759ffa0d6cd62c588d07c881f351 not found: ID does not exist" Jul 13 22:26:52.516903 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:52.516876 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb"] Jul 13 22:26:52.521055 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:52.521034 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65448f9f-57dcd4d946-74wcb"] Jul 13 22:26:53.316024 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:53.315992 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad9adcb4-d3cf-48de-9f91-aace8fac975a" path="/var/lib/kubelet/pods/ad9adcb4-d3cf-48de-9f91-aace8fac975a/volumes" Jul 13 22:26:57.156873 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:57.156821 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:57.255442 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:57.255411 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:57.384412 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:57.384380 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:57.639245 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:57.639218 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:57.740058 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:57.740029 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:58.646416 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.646383 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:58.696760 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.696726 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf"] Jul 13 22:26:58.697112 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.697097 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad9adcb4-d3cf-48de-9f91-aace8fac975a" containerName="maas-api" Jul 13 22:26:58.697112 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.697112 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad9adcb4-d3cf-48de-9f91-aace8fac975a" containerName="maas-api" Jul 13 22:26:58.697240 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.697188 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad9adcb4-d3cf-48de-9f91-aace8fac975a" containerName="maas-api" Jul 13 22:26:58.701462 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.701441 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:58.703456 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.703436 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a399a528-serving-cert\"" Jul 13 22:26:58.713228 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.713207 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf"] Jul 13 22:26:58.740415 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.740392 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ef2c481-8ad4-4d17-ac71-47106a9ea215-maas-api-tls\") pod \"maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:58.740564 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.740431 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkcq5\" (UniqueName: \"kubernetes.io/projected/2ef2c481-8ad4-4d17-ac71-47106a9ea215-kube-api-access-mkcq5\") pod \"maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:58.740564 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.740458 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef2c481-8ad4-4d17-ac71-47106a9ea215-openshift-service-ca\") pod \"maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:58.776430 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.776398 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:58.841788 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.841753 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcq5\" (UniqueName: \"kubernetes.io/projected/2ef2c481-8ad4-4d17-ac71-47106a9ea215-kube-api-access-mkcq5\") pod \"maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:58.842006 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.841803 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef2c481-8ad4-4d17-ac71-47106a9ea215-openshift-service-ca\") pod \"maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:58.842006 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.841903 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ef2c481-8ad4-4d17-ac71-47106a9ea215-maas-api-tls\") pod \"maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:58.842006 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:26:58.841986 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-serving-cert: secret "maas-api-e2e-ait-a399a528-serving-cert" not found Jul 13 22:26:58.842169 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:26:58.842065 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef2c481-8ad4-4d17-ac71-47106a9ea215-maas-api-tls podName:2ef2c481-8ad4-4d17-ac71-47106a9ea215 nodeName:}" failed. No retries permitted until 2026-07-13 22:26:59.342048971 +0000 UTC m=+1628.524893179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2ef2c481-8ad4-4d17-ac71-47106a9ea215-maas-api-tls") pod "maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" (UID: "2ef2c481-8ad4-4d17-ac71-47106a9ea215") : secret "maas-api-e2e-ait-a399a528-serving-cert" not found Jul 13 22:26:58.842524 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.842507 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef2c481-8ad4-4d17-ac71-47106a9ea215-openshift-service-ca\") pod \"maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:58.849735 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.849711 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkcq5\" (UniqueName: \"kubernetes.io/projected/2ef2c481-8ad4-4d17-ac71-47106a9ea215-kube-api-access-mkcq5\") pod \"maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:58.929764 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:58.929689 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:59.346962 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:59.346868 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ef2c481-8ad4-4d17-ac71-47106a9ea215-maas-api-tls\") pod \"maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:59.349269 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:59.349244 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ef2c481-8ad4-4d17-ac71-47106a9ea215-maas-api-tls\") pod \"maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:59.418903 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:59.418842 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:59.517489 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:59.517451 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:26:59.612315 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:59.612292 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:26:59.757191 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:26:59.757162 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf"] Jul 13 22:26:59.758953 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:26:59.758924 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ef2c481_8ad4_4d17_ac71_47106a9ea215.slice/crio-c7ce16ead1139b27f5021b094a732edefb49cfe3f7a2a00d606975873712c1f6 WatchSource:0}: Error finding container c7ce16ead1139b27f5021b094a732edefb49cfe3f7a2a00d606975873712c1f6: Status 404 returned error can't find the container with id c7ce16ead1139b27f5021b094a732edefb49cfe3f7a2a00d606975873712c1f6 Jul 13 22:27:00.536032 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:00.535936 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" event={"ID":"2ef2c481-8ad4-4d17-ac71-47106a9ea215","Type":"ContainerStarted","Data":"e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348"} Jul 13 22:27:00.536032 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:00.535990 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" event={"ID":"2ef2c481-8ad4-4d17-ac71-47106a9ea215","Type":"ContainerStarted","Data":"c7ce16ead1139b27f5021b094a732edefb49cfe3f7a2a00d606975873712c1f6"} Jul 13 22:27:00.536246 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:00.536067 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:27:00.557238 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:00.557190 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" podStartSLOduration=2.163308641 podStartE2EDuration="2.557174872s" totalImagesPullingTime="393.866231ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:27:00.554703506 +0000 UTC m=+1629.737547768" watchObservedRunningTime="2026-07-13 22:27:00.557174872 +0000 UTC m=+1629.740019103" Jul 13 22:27:00.963641 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:00.963613 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:27:06.545763 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:06.545738 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:27:14.593778 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:14.593739 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:27:26.327334 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:26.327298 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:27:49.263041 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:49.263009 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:27:49.357032 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:49.356995 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:27:54.217340 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:54.217304 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:27:54.316333 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:27:54.316305 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:04.139096 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.139059 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf"] Jul 13 22:28:04.139525 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.139330 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" podUID="2ef2c481-8ad4-4d17-ac71-47106a9ea215" containerName="maas-api" containerID="cri-o://e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348" gracePeriod=30 Jul 13 22:28:04.172176 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.172149 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:04.384417 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.384395 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:28:04.519493 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.519413 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ef2c481-8ad4-4d17-ac71-47106a9ea215-maas-api-tls\") pod \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " Jul 13 22:28:04.519638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.519522 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef2c481-8ad4-4d17-ac71-47106a9ea215-openshift-service-ca\") pod \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " Jul 13 22:28:04.519638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.519567 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkcq5\" (UniqueName: \"kubernetes.io/projected/2ef2c481-8ad4-4d17-ac71-47106a9ea215-kube-api-access-mkcq5\") pod \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\" (UID: \"2ef2c481-8ad4-4d17-ac71-47106a9ea215\") " Jul 13 22:28:04.519945 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.519922 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ef2c481-8ad4-4d17-ac71-47106a9ea215-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2ef2c481-8ad4-4d17-ac71-47106a9ea215" (UID: "2ef2c481-8ad4-4d17-ac71-47106a9ea215"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:28:04.521461 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.521440 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ef2c481-8ad4-4d17-ac71-47106a9ea215-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2ef2c481-8ad4-4d17-ac71-47106a9ea215" (UID: "2ef2c481-8ad4-4d17-ac71-47106a9ea215"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:28:04.521621 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.521595 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ef2c481-8ad4-4d17-ac71-47106a9ea215-kube-api-access-mkcq5" (OuterVolumeSpecName: "kube-api-access-mkcq5") pod "2ef2c481-8ad4-4d17-ac71-47106a9ea215" (UID: "2ef2c481-8ad4-4d17-ac71-47106a9ea215"). InnerVolumeSpecName "kube-api-access-mkcq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:28:04.620156 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.620129 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef2c481-8ad4-4d17-ac71-47106a9ea215-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:28:04.620156 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.620157 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkcq5\" (UniqueName: \"kubernetes.io/projected/2ef2c481-8ad4-4d17-ac71-47106a9ea215-kube-api-access-mkcq5\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:28:04.620325 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.620167 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ef2c481-8ad4-4d17-ac71-47106a9ea215-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:28:04.842876 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.842774 2570 generic.go:356] "Generic (PLEG): container finished" podID="2ef2c481-8ad4-4d17-ac71-47106a9ea215" containerID="e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348" exitCode=0 Jul 13 22:28:04.842876 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.842828 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" Jul 13 22:28:04.843118 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.842876 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" event={"ID":"2ef2c481-8ad4-4d17-ac71-47106a9ea215","Type":"ContainerDied","Data":"e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348"} Jul 13 22:28:04.843118 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.842966 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf" event={"ID":"2ef2c481-8ad4-4d17-ac71-47106a9ea215","Type":"ContainerDied","Data":"c7ce16ead1139b27f5021b094a732edefb49cfe3f7a2a00d606975873712c1f6"} Jul 13 22:28:04.843118 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.843000 2570 scope.go:117] "RemoveContainer" containerID="e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348" Jul 13 22:28:04.852260 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.852239 2570 scope.go:117] "RemoveContainer" containerID="e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348" Jul 13 22:28:04.852507 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:28:04.852488 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348\": container with ID starting with e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348 not found: ID does not exist" containerID="e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348" Jul 13 22:28:04.852563 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.852515 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348"} err="failed to get container status \"e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348\": rpc error: code = NotFound desc = could not find container \"e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348\": container with ID starting with e9b77579ea4ca523a2af2b2cdb82a9ccc87e6b4c924effe57a55c2fdbd308348 not found: ID does not exist" Jul 13 22:28:04.865109 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.865085 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf"] Jul 13 22:28:04.868877 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:04.868845 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a399a528-7c9d6784c7-pkqbf"] Jul 13 22:28:05.317354 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:05.317323 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ef2c481-8ad4-4d17-ac71-47106a9ea215" path="/var/lib/kubelet/pods/2ef2c481-8ad4-4d17-ac71-47106a9ea215/volumes" Jul 13 22:28:08.723721 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:08.723678 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:15.076209 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:15.076125 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:15.177219 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:15.177187 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:19.730527 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:19.730476 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:20.053660 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:20.053584 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:21.198336 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.198304 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:21.294931 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.294901 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:21.417188 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.417158 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg"] Jul 13 22:28:21.417663 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.417645 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef2c481-8ad4-4d17-ac71-47106a9ea215" containerName="maas-api" Jul 13 22:28:21.417717 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.417666 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef2c481-8ad4-4d17-ac71-47106a9ea215" containerName="maas-api" Jul 13 22:28:21.417794 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.417729 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef2c481-8ad4-4d17-ac71-47106a9ea215" containerName="maas-api" Jul 13 22:28:21.420937 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.420905 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:21.425142 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.425125 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-0081b233-serving-cert\"" Jul 13 22:28:21.439108 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.439086 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg"] Jul 13 22:28:21.467709 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.467652 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:21.566236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.566206 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-maas-api-tls\") pod \"maas-api-e2e-derive-0081b233-7997798577-b7ddg\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:21.566236 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.566239 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q7j8\" (UniqueName: \"kubernetes.io/projected/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-kube-api-access-5q7j8\") pod \"maas-api-e2e-derive-0081b233-7997798577-b7ddg\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:21.566407 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.566259 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-openshift-service-ca\") pod \"maas-api-e2e-derive-0081b233-7997798577-b7ddg\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:21.571386 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.571343 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:21.666910 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.666885 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-maas-api-tls\") pod \"maas-api-e2e-derive-0081b233-7997798577-b7ddg\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:21.667087 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.666920 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q7j8\" (UniqueName: \"kubernetes.io/projected/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-kube-api-access-5q7j8\") pod \"maas-api-e2e-derive-0081b233-7997798577-b7ddg\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:21.667087 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.666939 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-openshift-service-ca\") pod \"maas-api-e2e-derive-0081b233-7997798577-b7ddg\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:21.667563 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.667538 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-openshift-service-ca\") pod \"maas-api-e2e-derive-0081b233-7997798577-b7ddg\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:21.669210 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.669193 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-maas-api-tls\") pod \"maas-api-e2e-derive-0081b233-7997798577-b7ddg\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:21.679140 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.679119 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q7j8\" (UniqueName: \"kubernetes.io/projected/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-kube-api-access-5q7j8\") pod \"maas-api-e2e-derive-0081b233-7997798577-b7ddg\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:21.731426 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.731369 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:21.879540 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.879517 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg"] Jul 13 22:28:21.881746 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:28:21.881717 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c9b7700_9b6a_4d9f_a0de_b6451d43502d.slice/crio-fcb721e7182f3f13f85bf687ee1b0dea8d9438122cdc6a7c02404fc84ef4fb91 WatchSource:0}: Error finding container fcb721e7182f3f13f85bf687ee1b0dea8d9438122cdc6a7c02404fc84ef4fb91: Status 404 returned error can't find the container with id fcb721e7182f3f13f85bf687ee1b0dea8d9438122cdc6a7c02404fc84ef4fb91 Jul 13 22:28:21.920798 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:21.920769 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" event={"ID":"5c9b7700-9b6a-4d9f-a0de-b6451d43502d","Type":"ContainerStarted","Data":"fcb721e7182f3f13f85bf687ee1b0dea8d9438122cdc6a7c02404fc84ef4fb91"} Jul 13 22:28:22.246949 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:22.246918 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:22.369579 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:22.369557 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:22.926799 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:22.926763 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" event={"ID":"5c9b7700-9b6a-4d9f-a0de-b6451d43502d","Type":"ContainerStarted","Data":"e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2"} Jul 13 22:28:22.926979 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:22.926897 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:22.970832 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:22.970789 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" podStartSLOduration=1.541064666 podStartE2EDuration="1.970772356s" totalImagesPullingTime="429.70769ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:28:22.969968484 +0000 UTC m=+1712.152812716" watchObservedRunningTime="2026-07-13 22:28:22.970772356 +0000 UTC m=+1712.153616586" Jul 13 22:28:26.166264 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:26.166212 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:26.265730 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:26.265685 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:28:28.936605 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:28.936577 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:28:29.367553 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:28:29.367524 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:36.162890 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.162839 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg"] Jul 13 22:29:36.163413 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.163108 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" podUID="5c9b7700-9b6a-4d9f-a0de-b6451d43502d" containerName="maas-api" containerID="cri-o://e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2" gracePeriod=30 Jul 13 22:29:36.188127 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.188097 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:36.417967 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.417908 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:29:36.504332 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.504301 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-maas-api-tls\") pod \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " Jul 13 22:29:36.504531 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.504412 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-openshift-service-ca\") pod \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " Jul 13 22:29:36.504531 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.504460 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q7j8\" (UniqueName: \"kubernetes.io/projected/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-kube-api-access-5q7j8\") pod \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\" (UID: \"5c9b7700-9b6a-4d9f-a0de-b6451d43502d\") " Jul 13 22:29:36.504755 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.504732 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5c9b7700-9b6a-4d9f-a0de-b6451d43502d" (UID: "5c9b7700-9b6a-4d9f-a0de-b6451d43502d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:29:36.506436 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.506414 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5c9b7700-9b6a-4d9f-a0de-b6451d43502d" (UID: "5c9b7700-9b6a-4d9f-a0de-b6451d43502d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:29:36.506527 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.506416 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-kube-api-access-5q7j8" (OuterVolumeSpecName: "kube-api-access-5q7j8") pod "5c9b7700-9b6a-4d9f-a0de-b6451d43502d" (UID: "5c9b7700-9b6a-4d9f-a0de-b6451d43502d"). InnerVolumeSpecName "kube-api-access-5q7j8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:29:36.605465 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.605428 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:29:36.605465 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.605457 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q7j8\" (UniqueName: \"kubernetes.io/projected/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-kube-api-access-5q7j8\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:29:36.605465 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:36.605472 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c9b7700-9b6a-4d9f-a0de-b6451d43502d-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:29:37.273459 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:37.273426 2570 generic.go:356] "Generic (PLEG): container finished" podID="5c9b7700-9b6a-4d9f-a0de-b6451d43502d" containerID="e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2" exitCode=0 Jul 13 22:29:37.273459 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:37.273462 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" Jul 13 22:29:37.273964 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:37.273509 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" event={"ID":"5c9b7700-9b6a-4d9f-a0de-b6451d43502d","Type":"ContainerDied","Data":"e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2"} Jul 13 22:29:37.273964 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:37.273560 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg" event={"ID":"5c9b7700-9b6a-4d9f-a0de-b6451d43502d","Type":"ContainerDied","Data":"fcb721e7182f3f13f85bf687ee1b0dea8d9438122cdc6a7c02404fc84ef4fb91"} Jul 13 22:29:37.273964 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:37.273586 2570 scope.go:117] "RemoveContainer" containerID="e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2" Jul 13 22:29:37.282293 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:37.282274 2570 scope.go:117] "RemoveContainer" containerID="e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2" Jul 13 22:29:37.282539 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:29:37.282520 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2\": container with ID starting with e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2 not found: ID does not exist" containerID="e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2" Jul 13 22:29:37.282593 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:37.282547 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2"} err="failed to get container status \"e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2\": rpc error: code = NotFound desc = could not find container \"e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2\": container with ID starting with e2eb922d0150468d23c41b904f995b39da83ea8a9782d7d91af716d6cf0a6fe2 not found: ID does not exist" Jul 13 22:29:37.294500 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:37.294480 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg"] Jul 13 22:29:37.298457 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:37.298439 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0081b233-7997798577-b7ddg"] Jul 13 22:29:37.316795 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:37.316770 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c9b7700-9b6a-4d9f-a0de-b6451d43502d" path="/var/lib/kubelet/pods/5c9b7700-9b6a-4d9f-a0de-b6451d43502d/volumes" Jul 13 22:29:40.767536 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:40.767505 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:47.249442 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:47.249373 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:47.370448 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:47.370412 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:50.956369 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:50.956328 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:51.318684 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:51.318600 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:29:51.326696 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:51.326672 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:29:51.566045 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:51.566003 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:51.653678 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:51.653649 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:51.754255 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:51.754220 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:53.176858 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:53.176820 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:57.331997 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:57.331967 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:58.988201 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:58.988169 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:59.096760 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:59.096678 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:59.193351 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:59.193321 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:59.289505 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:59.289480 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:29:59.697124 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:29:59.697092 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:10.483640 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:10.483599 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:25.650441 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:25.650410 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:31.479502 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:31.479465 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:31.541450 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:31.541418 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:31.626096 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:31.626064 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:31.723253 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:31.723226 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:40.438104 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:40.438075 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:47.534738 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:47.534699 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:53.196757 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:53.196730 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:53.264082 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:53.264056 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:53.346573 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:53.346540 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:30:53.444903 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:30:53.444843 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:03.922527 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:03.922491 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:04.784440 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:04.784399 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:10.973368 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:10.973334 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:11.079055 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:11.079018 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:11.190494 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:11.190466 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:11.290126 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:11.290047 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:12.016908 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:12.016843 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:12.117072 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:12.117046 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:12.220029 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:12.219981 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:13.805746 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:13.805674 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:19.988328 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:19.988298 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:20.049125 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:20.049099 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:20.650091 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:20.650044 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:26.680942 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:26.680909 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:26.762605 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:26.762573 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:26.928725 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:26.928687 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:27.305033 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:27.304993 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:27.817047 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:27.817010 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:30.511323 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.511286 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:30.608087 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.608043 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:30.608776 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.608751 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv"] Jul 13 22:31:30.609309 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.609293 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9b7700-9b6a-4d9f-a0de-b6451d43502d" containerName="maas-api" Jul 13 22:31:30.609371 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.609312 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9b7700-9b6a-4d9f-a0de-b6451d43502d" containerName="maas-api" Jul 13 22:31:30.609410 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.609403 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c9b7700-9b6a-4d9f-a0de-b6451d43502d" containerName="maas-api" Jul 13 22:31:30.612664 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.612644 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:30.616223 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.616206 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2fa7da44-serving-cert\"" Jul 13 22:31:30.623695 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.623673 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv"] Jul 13 22:31:30.687597 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.687569 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:30.692088 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.692067 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-maas-api-tls\") pod \"maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:30.692196 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.692095 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-openshift-service-ca\") pod \"maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:30.692196 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.692123 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79npd\" (UniqueName: \"kubernetes.io/projected/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-kube-api-access-79npd\") pod \"maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:30.793364 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.793281 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-maas-api-tls\") pod \"maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:30.793364 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.793318 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-openshift-service-ca\") pod \"maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:30.793364 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.793344 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79npd\" (UniqueName: \"kubernetes.io/projected/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-kube-api-access-79npd\") pod \"maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:30.793664 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:31:30.793449 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-serving-cert: secret "maas-api-e2e-mt-2fa7da44-serving-cert" not found Jul 13 22:31:30.793664 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:31:30.793539 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-maas-api-tls podName:3c5bb2a9-7e11-4f11-8196-8716a075b2b1 nodeName:}" failed. No retries permitted until 2026-07-13 22:31:31.293520105 +0000 UTC m=+1900.476364314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-maas-api-tls") pod "maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" (UID: "3c5bb2a9-7e11-4f11-8196-8716a075b2b1") : secret "maas-api-e2e-mt-2fa7da44-serving-cert" not found Jul 13 22:31:30.793930 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.793913 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-openshift-service-ca\") pod \"maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:30.802057 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.802031 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79npd\" (UniqueName: \"kubernetes.io/projected/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-kube-api-access-79npd\") pod \"maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:30.861828 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:30.861803 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:31.205512 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:31.205483 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:31.297845 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:31.297793 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-maas-api-tls\") pod \"maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:31.298523 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:31.298474 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:31.300282 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:31.300260 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-maas-api-tls\") pod \"maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:31.523053 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:31.522980 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:31.655075 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:31.655045 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv"] Jul 13 22:31:31.659008 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:31:31.658980 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5bb2a9_7e11_4f11_8196_8716a075b2b1.slice/crio-b62fc0e450fcd3be596fe6919df328a2b118fe5cf16c4d20782aefeb755a429a WatchSource:0}: Error finding container b62fc0e450fcd3be596fe6919df328a2b118fe5cf16c4d20782aefeb755a429a: Status 404 returned error can't find the container with id b62fc0e450fcd3be596fe6919df328a2b118fe5cf16c4d20782aefeb755a429a Jul 13 22:31:31.660494 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:31.660475 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 22:31:31.786124 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:31.786037 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" event={"ID":"3c5bb2a9-7e11-4f11-8196-8716a075b2b1","Type":"ContainerStarted","Data":"b62fc0e450fcd3be596fe6919df328a2b118fe5cf16c4d20782aefeb755a429a"} Jul 13 22:31:32.793313 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:32.793275 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" event={"ID":"3c5bb2a9-7e11-4f11-8196-8716a075b2b1","Type":"ContainerStarted","Data":"78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b"} Jul 13 22:31:32.793704 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:32.793444 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:32.810759 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:32.810708 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" podStartSLOduration=2.348586953 podStartE2EDuration="2.810692501s" totalImagesPullingTime="462.105548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:31:32.808451211 +0000 UTC m=+1901.991295441" watchObservedRunningTime="2026-07-13 22:31:32.810692501 +0000 UTC m=+1901.993536732" Jul 13 22:31:34.976426 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:34.976393 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:35.644502 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:35.644454 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:35.739669 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:35.739636 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:36.053630 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:36.053553 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:38.803602 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:38.803577 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:41.861322 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:41.861288 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:43.242104 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:43.242053 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:52.850171 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:52.850131 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv"] Jul 13 22:31:52.850679 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:52.850382 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" podUID="3c5bb2a9-7e11-4f11-8196-8716a075b2b1" containerName="maas-api" containerID="cri-o://78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b" gracePeriod=30 Jul 13 22:31:52.894977 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:52.894946 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:31:53.101660 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.101602 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:53.183514 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.183482 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79npd\" (UniqueName: \"kubernetes.io/projected/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-kube-api-access-79npd\") pod \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " Jul 13 22:31:53.183682 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.183526 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-openshift-service-ca\") pod \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " Jul 13 22:31:53.183682 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.183555 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-maas-api-tls\") pod \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\" (UID: \"3c5bb2a9-7e11-4f11-8196-8716a075b2b1\") " Jul 13 22:31:53.184009 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.183982 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c5bb2a9-7e11-4f11-8196-8716a075b2b1" (UID: "3c5bb2a9-7e11-4f11-8196-8716a075b2b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:31:53.185654 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.185632 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-kube-api-access-79npd" (OuterVolumeSpecName: "kube-api-access-79npd") pod "3c5bb2a9-7e11-4f11-8196-8716a075b2b1" (UID: "3c5bb2a9-7e11-4f11-8196-8716a075b2b1"). InnerVolumeSpecName "kube-api-access-79npd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:31:53.185706 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.185632 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c5bb2a9-7e11-4f11-8196-8716a075b2b1" (UID: "3c5bb2a9-7e11-4f11-8196-8716a075b2b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:31:53.284794 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.284761 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79npd\" (UniqueName: \"kubernetes.io/projected/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-kube-api-access-79npd\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:31:53.284794 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.284788 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:31:53.284794 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.284799 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5bb2a9-7e11-4f11-8196-8716a075b2b1-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:31:53.889818 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.889786 2570 generic.go:356] "Generic (PLEG): container finished" podID="3c5bb2a9-7e11-4f11-8196-8716a075b2b1" containerID="78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b" exitCode=0 Jul 13 22:31:53.890301 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.889829 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" event={"ID":"3c5bb2a9-7e11-4f11-8196-8716a075b2b1","Type":"ContainerDied","Data":"78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b"} Jul 13 22:31:53.890301 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.889888 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" event={"ID":"3c5bb2a9-7e11-4f11-8196-8716a075b2b1","Type":"ContainerDied","Data":"b62fc0e450fcd3be596fe6919df328a2b118fe5cf16c4d20782aefeb755a429a"} Jul 13 22:31:53.890301 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.889910 2570 scope.go:117] "RemoveContainer" containerID="78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b" Jul 13 22:31:53.890301 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.889831 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv" Jul 13 22:31:53.897972 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.897959 2570 scope.go:117] "RemoveContainer" containerID="78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b" Jul 13 22:31:53.898203 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:31:53.898184 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b\": container with ID starting with 78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b not found: ID does not exist" containerID="78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b" Jul 13 22:31:53.898275 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.898216 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b"} err="failed to get container status \"78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b\": rpc error: code = NotFound desc = could not find container \"78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b\": container with ID starting with 78f58c1c088b45a1b66f07966d5c4ad406f9c5bb31e68a0070020ff8d897ee4b not found: ID does not exist" Jul 13 22:31:53.907108 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.907086 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv"] Jul 13 22:31:53.913463 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:53.913444 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa7da44-7f858f44fc-f4cjv"] Jul 13 22:31:55.316635 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:55.316600 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c5bb2a9-7e11-4f11-8196-8716a075b2b1" path="/var/lib/kubelet/pods/3c5bb2a9-7e11-4f11-8196-8716a075b2b1/volumes" Jul 13 22:31:57.330042 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:31:57.329998 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:03.253179 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:03.253148 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:03.343191 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:03.343163 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:08.003051 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:08.003009 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:08.229977 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:08.229929 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:08.349055 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:08.348951 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:08.448865 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:08.448827 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:08.570030 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:08.569974 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:14.249491 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:14.249452 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:14.362963 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:14.362936 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:14.449079 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:14.449050 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:14.588273 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:14.588195 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:20.168164 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:20.168131 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:20.233332 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:20.233306 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:26.935626 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:26.935591 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:27.237519 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:27.237437 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:27.627800 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:27.627766 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:29.840520 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:29.840488 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:29.917742 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:29.917710 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq"] Jul 13 22:32:29.918117 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:29.918098 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c5bb2a9-7e11-4f11-8196-8716a075b2b1" containerName="maas-api" Jul 13 22:32:29.918117 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:29.918118 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c5bb2a9-7e11-4f11-8196-8716a075b2b1" containerName="maas-api" Jul 13 22:32:29.918235 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:29.918176 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c5bb2a9-7e11-4f11-8196-8716a075b2b1" containerName="maas-api" Jul 13 22:32:29.921166 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:29.921151 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:29.923116 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:29.923096 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-86e22e9e-serving-cert\"" Jul 13 22:32:29.924447 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:29.924421 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:29.930840 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:29.930821 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq"] Jul 13 22:32:29.992730 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:29.992706 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:30.086978 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.086951 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsjqw\" (UniqueName: \"kubernetes.io/projected/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-kube-api-access-lsjqw\") pod \"maas-api-e2e-mt-86e22e9e-66875688cb-fhskq\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:30.087128 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.086999 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-openshift-service-ca\") pod \"maas-api-e2e-mt-86e22e9e-66875688cb-fhskq\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:30.087128 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.087025 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-maas-api-tls\") pod \"maas-api-e2e-mt-86e22e9e-66875688cb-fhskq\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:30.122707 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.122681 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:30.187956 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.187928 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsjqw\" (UniqueName: \"kubernetes.io/projected/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-kube-api-access-lsjqw\") pod \"maas-api-e2e-mt-86e22e9e-66875688cb-fhskq\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:30.188088 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.187987 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-openshift-service-ca\") pod \"maas-api-e2e-mt-86e22e9e-66875688cb-fhskq\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:30.188088 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.188015 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-maas-api-tls\") pod \"maas-api-e2e-mt-86e22e9e-66875688cb-fhskq\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:30.188604 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.188587 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-openshift-service-ca\") pod \"maas-api-e2e-mt-86e22e9e-66875688cb-fhskq\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:30.190334 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.190315 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-maas-api-tls\") pod \"maas-api-e2e-mt-86e22e9e-66875688cb-fhskq\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:30.195188 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.195158 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsjqw\" (UniqueName: \"kubernetes.io/projected/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-kube-api-access-lsjqw\") pod \"maas-api-e2e-mt-86e22e9e-66875688cb-fhskq\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:30.231963 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.231945 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:30.382693 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.382622 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq"] Jul 13 22:32:30.387223 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:32:30.387196 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c0b451_ddc8_4f9f_ac94_7a02d3be0e6e.slice/crio-d8781a135618a830b9ec284e914344872a110e4cd132db9fbed727978578ba05 WatchSource:0}: Error finding container d8781a135618a830b9ec284e914344872a110e4cd132db9fbed727978578ba05: Status 404 returned error can't find the container with id d8781a135618a830b9ec284e914344872a110e4cd132db9fbed727978578ba05 Jul 13 22:32:30.610870 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.610821 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:30.711598 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:30.711518 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:31.078655 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:31.078555 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" event={"ID":"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e","Type":"ContainerStarted","Data":"d1c78df40d840c888fa1610fd838e69639bd32f37c5b3107e74af894a6b35706"} Jul 13 22:32:31.078655 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:31.078640 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" event={"ID":"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e","Type":"ContainerStarted","Data":"d8781a135618a830b9ec284e914344872a110e4cd132db9fbed727978578ba05"} Jul 13 22:32:31.079094 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:31.078662 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:31.096242 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:31.096197 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" podStartSLOduration=1.76872684 podStartE2EDuration="2.096183048s" totalImagesPullingTime="327.456208ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:32:31.094355318 +0000 UTC m=+1960.277199548" watchObservedRunningTime="2026-07-13 22:32:31.096183048 +0000 UTC m=+1960.279027279" Jul 13 22:32:34.320119 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:34.320087 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:36.503645 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:36.503612 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:37.089749 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:37.089719 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:32:50.032928 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:50.032817 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:50.134882 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:50.134809 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:51.506320 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:51.506290 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:51.608810 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:51.608776 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:32:58.288640 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:32:58.288605 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:33:28.202073 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.202037 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq"] Jul 13 22:33:28.202607 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.202300 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" podUID="a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e" containerName="maas-api" containerID="cri-o://d1c78df40d840c888fa1610fd838e69639bd32f37c5b3107e74af894a6b35706" gracePeriod=30 Jul 13 22:33:28.236923 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.236893 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:33:28.339208 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.339174 2570 generic.go:356] "Generic (PLEG): container finished" podID="a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e" containerID="d1c78df40d840c888fa1610fd838e69639bd32f37c5b3107e74af894a6b35706" exitCode=0 Jul 13 22:33:28.339346 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.339249 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" event={"ID":"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e","Type":"ContainerDied","Data":"d1c78df40d840c888fa1610fd838e69639bd32f37c5b3107e74af894a6b35706"} Jul 13 22:33:28.448583 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.448558 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:33:28.590789 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.590715 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-openshift-service-ca\") pod \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " Jul 13 22:33:28.590789 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.590748 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-maas-api-tls\") pod \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " Jul 13 22:33:28.590789 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.590770 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsjqw\" (UniqueName: \"kubernetes.io/projected/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-kube-api-access-lsjqw\") pod \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\" (UID: \"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e\") " Jul 13 22:33:28.591163 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.591137 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e" (UID: "a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:33:28.592866 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.592831 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e" (UID: "a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:33:28.592960 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.592940 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-kube-api-access-lsjqw" (OuterVolumeSpecName: "kube-api-access-lsjqw") pod "a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e" (UID: "a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e"). InnerVolumeSpecName "kube-api-access-lsjqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:33:28.691888 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.691840 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:33:28.691888 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.691884 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:33:28.691888 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:28.691894 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsjqw\" (UniqueName: \"kubernetes.io/projected/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e-kube-api-access-lsjqw\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:33:29.345274 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:29.345247 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" Jul 13 22:33:29.345611 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:29.345240 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq" event={"ID":"a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e","Type":"ContainerDied","Data":"d8781a135618a830b9ec284e914344872a110e4cd132db9fbed727978578ba05"} Jul 13 22:33:29.345611 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:29.345375 2570 scope.go:117] "RemoveContainer" containerID="d1c78df40d840c888fa1610fd838e69639bd32f37c5b3107e74af894a6b35706" Jul 13 22:33:29.361043 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:29.361021 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq"] Jul 13 22:33:29.365872 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:29.365836 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-86e22e9e-66875688cb-fhskq"] Jul 13 22:33:31.316890 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:31.316841 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e" path="/var/lib/kubelet/pods/a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e/volumes" Jul 13 22:33:32.717797 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:32.717764 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:33:37.761623 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:33:37.761589 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:14.625320 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:14.625232 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:14.872035 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:14.871989 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:20.421344 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:20.421313 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:32.183987 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:32.183949 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:34.665088 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:34.665062 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:34.727121 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:34.727087 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:44.919471 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:44.919426 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:45.157298 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:45.157267 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:46.271115 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:46.271083 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:46.409018 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:46.408986 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:47.115999 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:47.115958 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:47.218079 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:47.218018 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:47.329029 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:47.328993 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:48.614838 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:48.614810 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:51.343518 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:51.343492 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:34:51.352412 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:51.352379 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:34:54.591843 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:54.591813 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:54.668355 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:54.668328 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:34:55.248695 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:34:55.248663 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:01.332028 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:01.331998 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:01.436985 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:01.436950 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:01.530115 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:01.530081 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:12.044438 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:12.044404 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:12.392627 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:12.392599 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:33.313214 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:33.313176 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:39.247390 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:39.247355 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:39.347124 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:39.347096 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:39.615634 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:39.615605 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:39.924435 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:39.924349 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:40.152048 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:40.152019 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:43.048399 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.048304 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:43.128340 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.128306 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:43.128951 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.128928 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh"] Jul 13 22:35:43.129313 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.129298 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e" containerName="maas-api" Jul 13 22:35:43.129384 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.129316 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e" containerName="maas-api" Jul 13 22:35:43.129442 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.129425 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c0b451-ddc8-4f9f-ac94-7a02d3be0e6e" containerName="maas-api" Jul 13 22:35:43.132357 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.132338 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:43.134831 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.134807 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-7da020-serving-cert\"" Jul 13 22:35:43.145760 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.145742 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh"] Jul 13 22:35:43.194589 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.194542 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:43.294621 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.294589 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f68k\" (UniqueName: \"kubernetes.io/projected/e9d34f11-03c4-492c-9c15-96fae88e66d6-kube-api-access-6f68k\") pod \"maas-api-e2e-inf-a-7da020-6889499674-dqfsh\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:43.294788 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.294643 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9d34f11-03c4-492c-9c15-96fae88e66d6-maas-api-tls\") pod \"maas-api-e2e-inf-a-7da020-6889499674-dqfsh\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:43.294788 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.294714 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9d34f11-03c4-492c-9c15-96fae88e66d6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7da020-6889499674-dqfsh\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:43.295450 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.295431 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:43.395494 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.395461 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9d34f11-03c4-492c-9c15-96fae88e66d6-maas-api-tls\") pod \"maas-api-e2e-inf-a-7da020-6889499674-dqfsh\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:43.395660 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.395502 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9d34f11-03c4-492c-9c15-96fae88e66d6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7da020-6889499674-dqfsh\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:43.395660 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.395548 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f68k\" (UniqueName: \"kubernetes.io/projected/e9d34f11-03c4-492c-9c15-96fae88e66d6-kube-api-access-6f68k\") pod \"maas-api-e2e-inf-a-7da020-6889499674-dqfsh\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:43.396199 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.396171 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9d34f11-03c4-492c-9c15-96fae88e66d6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7da020-6889499674-dqfsh\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:43.397791 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.397775 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9d34f11-03c4-492c-9c15-96fae88e66d6-maas-api-tls\") pod \"maas-api-e2e-inf-a-7da020-6889499674-dqfsh\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:43.402979 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.402959 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f68k\" (UniqueName: \"kubernetes.io/projected/e9d34f11-03c4-492c-9c15-96fae88e66d6-kube-api-access-6f68k\") pod \"maas-api-e2e-inf-a-7da020-6889499674-dqfsh\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:43.442988 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.442964 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:43.569587 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.569562 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh"] Jul 13 22:35:43.571624 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:35:43.571585 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9d34f11_03c4_492c_9c15_96fae88e66d6.slice/crio-343c33893fab47272167673ea1af4343b89ea56583320429e2ffeb99a9394ba8 WatchSource:0}: Error finding container 343c33893fab47272167673ea1af4343b89ea56583320429e2ffeb99a9394ba8: Status 404 returned error can't find the container with id 343c33893fab47272167673ea1af4343b89ea56583320429e2ffeb99a9394ba8 Jul 13 22:35:43.707592 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.707513 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:43.808057 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.808021 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:43.947092 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:43.947060 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" event={"ID":"e9d34f11-03c4-492c-9c15-96fae88e66d6","Type":"ContainerStarted","Data":"343c33893fab47272167673ea1af4343b89ea56583320429e2ffeb99a9394ba8"} Jul 13 22:35:44.952565 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:44.952535 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" event={"ID":"e9d34f11-03c4-492c-9c15-96fae88e66d6","Type":"ContainerStarted","Data":"da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5"} Jul 13 22:35:44.952997 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:44.952610 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:44.969396 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:44.969353 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" podStartSLOduration=1.366500989 podStartE2EDuration="1.96934002s" totalImagesPullingTime="602.839031ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:35:44.968392124 +0000 UTC m=+2154.151236336" watchObservedRunningTime="2026-07-13 22:35:44.96934002 +0000 UTC m=+2154.152184253" Jul 13 22:35:47.458346 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:47.458314 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:47.782167 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:47.782078 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:48.103248 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:48.103168 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:48.390289 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:48.390257 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:50.962285 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:50.962258 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:35:51.036515 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.036483 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:51.121981 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.121952 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq"] Jul 13 22:35:51.125372 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.125355 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:51.129057 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.129037 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a502db-serving-cert\"" Jul 13 22:35:51.133490 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.133462 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:51.137433 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.137408 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq"] Jul 13 22:35:51.162429 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.162398 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx4dx\" (UniqueName: \"kubernetes.io/projected/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-kube-api-access-wx4dx\") pod \"maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:51.162568 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.162474 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-maas-api-tls\") pod \"maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:51.162568 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.162499 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:51.231228 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.231147 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:51.263221 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.263197 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx4dx\" (UniqueName: \"kubernetes.io/projected/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-kube-api-access-wx4dx\") pod \"maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:51.263356 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.263259 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-maas-api-tls\") pod \"maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:51.263356 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.263287 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:51.263918 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.263902 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:51.265638 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.265621 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a502db-serving-cert\"" Jul 13 22:35:51.272142 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.272124 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx4dx\" (UniqueName: \"kubernetes.io/projected/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-kube-api-access-wx4dx\") pod \"maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:51.273880 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:35:51.273859 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-serving-cert: secret "maas-api-e2e-inf-b-a502db-serving-cert" not found Jul 13 22:35:51.273979 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:35:51.273920 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-maas-api-tls podName:bebbcdb8-c887-4ace-86c1-2c8b32b9443c nodeName:}" failed. No retries permitted until 2026-07-13 22:35:51.773905817 +0000 UTC m=+2160.956750025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-maas-api-tls") pod "maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" (UID: "bebbcdb8-c887-4ace-86c1-2c8b32b9443c") : secret "maas-api-e2e-inf-b-a502db-serving-cert" not found Jul 13 22:35:51.661021 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.660987 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:51.762213 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.762181 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:51.863674 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.863641 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:51.868906 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.868877 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-maas-api-tls\") pod \"maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:51.871237 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:51.871210 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-maas-api-tls\") pod \"maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:52.036772 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:52.036679 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:52.166889 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:52.166834 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq"] Jul 13 22:35:52.168808 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:35:52.168779 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbebbcdb8_c887_4ace_86c1_2c8b32b9443c.slice/crio-e06c4cbacb289676919f45a8522e3020c92ca85c18edbf652aec1b79c499ced8 WatchSource:0}: Error finding container e06c4cbacb289676919f45a8522e3020c92ca85c18edbf652aec1b79c499ced8: Status 404 returned error can't find the container with id e06c4cbacb289676919f45a8522e3020c92ca85c18edbf652aec1b79c499ced8 Jul 13 22:35:52.991575 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:52.991541 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" event={"ID":"bebbcdb8-c887-4ace-86c1-2c8b32b9443c","Type":"ContainerStarted","Data":"577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342"} Jul 13 22:35:52.991761 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:52.991600 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" event={"ID":"bebbcdb8-c887-4ace-86c1-2c8b32b9443c","Type":"ContainerStarted","Data":"e06c4cbacb289676919f45a8522e3020c92ca85c18edbf652aec1b79c499ced8"} Jul 13 22:35:52.991761 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:52.991646 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:53.010032 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:53.009978 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" podStartSLOduration=1.691625572 podStartE2EDuration="2.009958452s" totalImagesPullingTime="318.33288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:35:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:35:53.007557532 +0000 UTC m=+2162.190401762" watchObservedRunningTime="2026-07-13 22:35:53.009958452 +0000 UTC m=+2162.192802684" Jul 13 22:35:55.509328 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:55.509298 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:35:59.001882 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:59.001836 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:35:59.279629 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:35:59.279541 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:36:04.645120 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:36:04.645084 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:36:04.745353 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:36:04.745311 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:36:13.549096 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:36:13.549062 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:36:13.646514 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:36:13.646480 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:36:18.580613 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:36:18.580573 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:36:18.681802 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:36:18.681778 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:36:28.485196 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:36:28.485163 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:36:49.084324 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:36:49.084295 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:36:50.214221 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:36:50.214194 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:36:50.512829 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:36:50.512742 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:36:50.620354 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:36:50.620320 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:00.421461 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.421419 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh"] Jul 13 22:37:00.421974 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.421638 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" podUID="e9d34f11-03c4-492c-9c15-96fae88e66d6" containerName="maas-api" containerID="cri-o://da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5" gracePeriod=30 Jul 13 22:37:00.449098 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.449072 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:00.671275 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.671251 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:37:00.768261 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.768177 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9d34f11-03c4-492c-9c15-96fae88e66d6-maas-api-tls\") pod \"e9d34f11-03c4-492c-9c15-96fae88e66d6\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " Jul 13 22:37:00.768261 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.768215 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9d34f11-03c4-492c-9c15-96fae88e66d6-openshift-service-ca\") pod \"e9d34f11-03c4-492c-9c15-96fae88e66d6\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " Jul 13 22:37:00.768458 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.768274 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f68k\" (UniqueName: \"kubernetes.io/projected/e9d34f11-03c4-492c-9c15-96fae88e66d6-kube-api-access-6f68k\") pod \"e9d34f11-03c4-492c-9c15-96fae88e66d6\" (UID: \"e9d34f11-03c4-492c-9c15-96fae88e66d6\") " Jul 13 22:37:00.768693 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.768667 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9d34f11-03c4-492c-9c15-96fae88e66d6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9d34f11-03c4-492c-9c15-96fae88e66d6" (UID: "e9d34f11-03c4-492c-9c15-96fae88e66d6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:37:00.770304 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.770280 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9d34f11-03c4-492c-9c15-96fae88e66d6-kube-api-access-6f68k" (OuterVolumeSpecName: "kube-api-access-6f68k") pod "e9d34f11-03c4-492c-9c15-96fae88e66d6" (UID: "e9d34f11-03c4-492c-9c15-96fae88e66d6"). InnerVolumeSpecName "kube-api-access-6f68k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:37:00.770304 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.770294 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9d34f11-03c4-492c-9c15-96fae88e66d6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9d34f11-03c4-492c-9c15-96fae88e66d6" (UID: "e9d34f11-03c4-492c-9c15-96fae88e66d6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:37:00.868872 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.868819 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9d34f11-03c4-492c-9c15-96fae88e66d6-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:37:00.868872 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.868866 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9d34f11-03c4-492c-9c15-96fae88e66d6-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:37:00.868872 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:00.868879 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f68k\" (UniqueName: \"kubernetes.io/projected/e9d34f11-03c4-492c-9c15-96fae88e66d6-kube-api-access-6f68k\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:37:01.301439 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:01.301402 2570 generic.go:356] "Generic (PLEG): container finished" podID="e9d34f11-03c4-492c-9c15-96fae88e66d6" containerID="da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5" exitCode=0 Jul 13 22:37:01.301619 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:01.301446 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" Jul 13 22:37:01.301619 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:01.301466 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" event={"ID":"e9d34f11-03c4-492c-9c15-96fae88e66d6","Type":"ContainerDied","Data":"da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5"} Jul 13 22:37:01.301619 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:01.301509 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh" event={"ID":"e9d34f11-03c4-492c-9c15-96fae88e66d6","Type":"ContainerDied","Data":"343c33893fab47272167673ea1af4343b89ea56583320429e2ffeb99a9394ba8"} Jul 13 22:37:01.301619 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:01.301530 2570 scope.go:117] "RemoveContainer" containerID="da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5" Jul 13 22:37:01.310664 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:01.310640 2570 scope.go:117] "RemoveContainer" containerID="da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5" Jul 13 22:37:01.310981 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:37:01.310960 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5\": container with ID starting with da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5 not found: ID does not exist" containerID="da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5" Jul 13 22:37:01.311058 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:01.310988 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5"} err="failed to get container status \"da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5\": rpc error: code = NotFound desc = could not find container \"da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5\": container with ID starting with da36525630189e41baebec7ebec6278cee73bf7ce3ea371582a66d4ce5141da5 not found: ID does not exist" Jul 13 22:37:01.321679 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:01.321652 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh"] Jul 13 22:37:01.327343 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:01.327306 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7da020-6889499674-dqfsh"] Jul 13 22:37:03.316600 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:03.316567 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9d34f11-03c4-492c-9c15-96fae88e66d6" path="/var/lib/kubelet/pods/e9d34f11-03c4-492c-9c15-96fae88e66d6/volumes" Jul 13 22:37:05.083319 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:05.083288 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:10.975991 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:10.975959 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:11.075987 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:11.075952 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:15.774449 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:15.774368 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:15.998887 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:15.998832 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:16.738521 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:16.738486 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:16.840409 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:16.840381 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:26.531734 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.531696 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq"] Jul 13 22:37:26.532136 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.531943 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" podUID="bebbcdb8-c887-4ace-86c1-2c8b32b9443c" containerName="maas-api" containerID="cri-o://577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342" gracePeriod=30 Jul 13 22:37:26.555235 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.555213 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:26.777924 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.777901 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:37:26.894710 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.894681 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx4dx\" (UniqueName: \"kubernetes.io/projected/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-kube-api-access-wx4dx\") pod \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " Jul 13 22:37:26.894916 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.894772 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-openshift-service-ca\") pod \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " Jul 13 22:37:26.894916 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.894882 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-maas-api-tls\") pod \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\" (UID: \"bebbcdb8-c887-4ace-86c1-2c8b32b9443c\") " Jul 13 22:37:26.895138 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.895114 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bebbcdb8-c887-4ace-86c1-2c8b32b9443c" (UID: "bebbcdb8-c887-4ace-86c1-2c8b32b9443c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:37:26.896751 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.896725 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bebbcdb8-c887-4ace-86c1-2c8b32b9443c" (UID: "bebbcdb8-c887-4ace-86c1-2c8b32b9443c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:37:26.896889 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.896813 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-kube-api-access-wx4dx" (OuterVolumeSpecName: "kube-api-access-wx4dx") pod "bebbcdb8-c887-4ace-86c1-2c8b32b9443c" (UID: "bebbcdb8-c887-4ace-86c1-2c8b32b9443c"). InnerVolumeSpecName "kube-api-access-wx4dx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:37:26.996134 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.996107 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:37:26.996134 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.996131 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:37:26.996275 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:26.996140 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx4dx\" (UniqueName: \"kubernetes.io/projected/bebbcdb8-c887-4ace-86c1-2c8b32b9443c-kube-api-access-wx4dx\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:37:27.423801 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:27.423766 2570 generic.go:356] "Generic (PLEG): container finished" podID="bebbcdb8-c887-4ace-86c1-2c8b32b9443c" containerID="577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342" exitCode=0 Jul 13 22:37:27.423968 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:27.423808 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" event={"ID":"bebbcdb8-c887-4ace-86c1-2c8b32b9443c","Type":"ContainerDied","Data":"577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342"} Jul 13 22:37:27.423968 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:27.423874 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" event={"ID":"bebbcdb8-c887-4ace-86c1-2c8b32b9443c","Type":"ContainerDied","Data":"e06c4cbacb289676919f45a8522e3020c92ca85c18edbf652aec1b79c499ced8"} Jul 13 22:37:27.423968 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:27.423814 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq" Jul 13 22:37:27.424078 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:27.423901 2570 scope.go:117] "RemoveContainer" containerID="577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342" Jul 13 22:37:27.432226 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:27.432202 2570 scope.go:117] "RemoveContainer" containerID="577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342" Jul 13 22:37:27.432472 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:37:27.432448 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342\": container with ID starting with 577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342 not found: ID does not exist" containerID="577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342" Jul 13 22:37:27.432527 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:27.432481 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342"} err="failed to get container status \"577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342\": rpc error: code = NotFound desc = could not find container \"577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342\": container with ID starting with 577bcfd05f4e58361be266bb84928a2576e3fd0db2efdef3e9c7600b7c024342 not found: ID does not exist" Jul 13 22:37:27.437356 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:27.437336 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq"] Jul 13 22:37:27.439911 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:27.439893 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a502db-75bfbfbf79-ft4zq"] Jul 13 22:37:29.316286 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:29.316256 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bebbcdb8-c887-4ace-86c1-2c8b32b9443c" path="/var/lib/kubelet/pods/bebbcdb8-c887-4ace-86c1-2c8b32b9443c/volumes" Jul 13 22:37:31.206342 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:31.206309 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:37.172895 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:37.172840 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:37.233035 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:37.233011 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:41.870148 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:41.870121 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:42.099498 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:42.099470 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:42.414011 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:42.413984 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:42.825678 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:42.825607 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:43.273829 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:43.273796 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:45.756615 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.756576 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:45.837450 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.837413 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk"] Jul 13 22:37:45.837822 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.837808 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bebbcdb8-c887-4ace-86c1-2c8b32b9443c" containerName="maas-api" Jul 13 22:37:45.837884 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.837824 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bebbcdb8-c887-4ace-86c1-2c8b32b9443c" containerName="maas-api" Jul 13 22:37:45.837884 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.837832 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9d34f11-03c4-492c-9c15-96fae88e66d6" containerName="maas-api" Jul 13 22:37:45.837884 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.837839 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9d34f11-03c4-492c-9c15-96fae88e66d6" containerName="maas-api" Jul 13 22:37:45.837981 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.837915 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9d34f11-03c4-492c-9c15-96fae88e66d6" containerName="maas-api" Jul 13 22:37:45.837981 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.837928 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bebbcdb8-c887-4ace-86c1-2c8b32b9443c" containerName="maas-api" Jul 13 22:37:45.842184 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.842154 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:45.844681 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.844659 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-fe823c-serving-cert\"" Jul 13 22:37:45.844919 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.844895 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:45.857539 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.857511 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk"] Jul 13 22:37:45.922158 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.922121 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:45.954869 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.954809 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4whgj\" (UniqueName: \"kubernetes.io/projected/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-kube-api-access-4whgj\") pod \"maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:45.955024 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.954875 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-maas-api-tls\") pod \"maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:45.955024 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:45.954959 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-openshift-service-ca\") pod \"maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:46.055779 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:46.055690 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4whgj\" (UniqueName: \"kubernetes.io/projected/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-kube-api-access-4whgj\") pod \"maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:46.055779 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:46.055734 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-maas-api-tls\") pod \"maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:46.055779 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:46.055770 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-openshift-service-ca\") pod \"maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:46.056101 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:37:46.055918 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-serving-cert: secret "maas-api-e2e-api-a-fe823c-serving-cert" not found Jul 13 22:37:46.056101 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:37:46.055996 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-maas-api-tls podName:7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46 nodeName:}" failed. No retries permitted until 2026-07-13 22:37:46.555976165 +0000 UTC m=+2275.738820377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-maas-api-tls") pod "maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" (UID: "7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46") : secret "maas-api-e2e-api-a-fe823c-serving-cert" not found Jul 13 22:37:46.056393 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:46.056376 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-openshift-service-ca\") pod \"maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:46.067577 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:46.067546 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4whgj\" (UniqueName: \"kubernetes.io/projected/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-kube-api-access-4whgj\") pod \"maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:46.382360 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:46.382331 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:46.479515 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:46.479485 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:46.561454 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:46.561418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-maas-api-tls\") pod \"maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:46.563779 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:46.563752 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-maas-api-tls\") pod \"maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:46.582407 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:46.582383 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:46.756043 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:46.755966 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:47.097009 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:47.096977 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk"] Jul 13 22:37:47.098300 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:37:47.098263 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5afa9a_9fc3_4d2c_8345_f3d1f1a07a46.slice/crio-82496e32d5f10d5afc1025e7bfe2c9fb556507e8eeed044c2eeaf1ecc35042bb WatchSource:0}: Error finding container 82496e32d5f10d5afc1025e7bfe2c9fb556507e8eeed044c2eeaf1ecc35042bb: Status 404 returned error can't find the container with id 82496e32d5f10d5afc1025e7bfe2c9fb556507e8eeed044c2eeaf1ecc35042bb Jul 13 22:37:47.099939 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:47.099920 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 22:37:47.517741 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:47.517712 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" event={"ID":"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46","Type":"ContainerStarted","Data":"82496e32d5f10d5afc1025e7bfe2c9fb556507e8eeed044c2eeaf1ecc35042bb"} Jul 13 22:37:48.524123 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:48.524086 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" event={"ID":"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46","Type":"ContainerStarted","Data":"11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa"} Jul 13 22:37:48.524494 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:48.524158 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:48.542495 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:48.542446 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" podStartSLOduration=3.08939792 podStartE2EDuration="3.5424315s" totalImagesPullingTime="453.03358ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:37:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:37:48.541507045 +0000 UTC m=+2277.724351303" watchObservedRunningTime="2026-07-13 22:37:48.5424315 +0000 UTC m=+2277.725275730" Jul 13 22:37:50.248454 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:50.248403 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:50.688326 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:50.688296 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:51.056729 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:51.056645 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:51.151177 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:51.151146 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:51.617050 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:51.617016 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:53.895557 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:53.895526 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:53.984208 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:53.984175 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw"] Jul 13 22:37:53.989243 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:53.989223 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:53.991253 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:53.991232 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-2a08d3-serving-cert\"" Jul 13 22:37:53.991338 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:53.991249 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:54.001689 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.001666 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw"] Jul 13 22:37:54.027175 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.027148 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28add1d8-2469-43a7-9217-961d441f1ddf-openshift-service-ca\") pod \"maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:54.027297 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.027184 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7l8s\" (UniqueName: \"kubernetes.io/projected/28add1d8-2469-43a7-9217-961d441f1ddf-kube-api-access-m7l8s\") pod \"maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:54.027347 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.027306 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28add1d8-2469-43a7-9217-961d441f1ddf-maas-api-tls\") pod \"maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:54.065069 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.065037 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:54.128611 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.128570 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28add1d8-2469-43a7-9217-961d441f1ddf-maas-api-tls\") pod \"maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:54.128611 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.128615 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28add1d8-2469-43a7-9217-961d441f1ddf-openshift-service-ca\") pod \"maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:54.128839 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.128641 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7l8s\" (UniqueName: \"kubernetes.io/projected/28add1d8-2469-43a7-9217-961d441f1ddf-kube-api-access-m7l8s\") pod \"maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:54.128839 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:37:54.128727 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-serving-cert: secret "maas-api-e2e-api-b-2a08d3-serving-cert" not found Jul 13 22:37:54.128839 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:37:54.128793 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28add1d8-2469-43a7-9217-961d441f1ddf-maas-api-tls podName:28add1d8-2469-43a7-9217-961d441f1ddf nodeName:}" failed. No retries permitted until 2026-07-13 22:37:54.628777354 +0000 UTC m=+2283.811621566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/28add1d8-2469-43a7-9217-961d441f1ddf-maas-api-tls") pod "maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" (UID: "28add1d8-2469-43a7-9217-961d441f1ddf") : secret "maas-api-e2e-api-b-2a08d3-serving-cert" not found Jul 13 22:37:54.129238 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.129219 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28add1d8-2469-43a7-9217-961d441f1ddf-openshift-service-ca\") pod \"maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:54.137509 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.137481 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7l8s\" (UniqueName: \"kubernetes.io/projected/28add1d8-2469-43a7-9217-961d441f1ddf-kube-api-access-m7l8s\") pod \"maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:54.467125 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.467093 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:54.534834 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.534812 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:37:54.555960 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.555927 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:54.612385 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.612339 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:37:54.633813 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.633778 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28add1d8-2469-43a7-9217-961d441f1ddf-maas-api-tls\") pod \"maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:54.636129 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.636100 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28add1d8-2469-43a7-9217-961d441f1ddf-maas-api-tls\") pod \"maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:54.900508 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:54.900480 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:55.035120 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:55.035095 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw"] Jul 13 22:37:55.036627 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:37:55.036599 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28add1d8_2469_43a7_9217_961d441f1ddf.slice/crio-d8f470659246877babc1fe32642f3eab663e6d5cb7fc2a1073b503b9556bdc0d WatchSource:0}: Error finding container d8f470659246877babc1fe32642f3eab663e6d5cb7fc2a1073b503b9556bdc0d: Status 404 returned error can't find the container with id d8f470659246877babc1fe32642f3eab663e6d5cb7fc2a1073b503b9556bdc0d Jul 13 22:37:55.558733 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:55.558698 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" event={"ID":"28add1d8-2469-43a7-9217-961d441f1ddf","Type":"ContainerStarted","Data":"25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21"} Jul 13 22:37:55.558903 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:55.558760 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" event={"ID":"28add1d8-2469-43a7-9217-961d441f1ddf","Type":"ContainerStarted","Data":"d8f470659246877babc1fe32642f3eab663e6d5cb7fc2a1073b503b9556bdc0d"} Jul 13 22:37:55.558903 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:55.558834 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:37:55.577227 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:55.577147 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" podStartSLOduration=2.162199201 podStartE2EDuration="2.57713225s" totalImagesPullingTime="414.933049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:37:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:37:55.576110955 +0000 UTC m=+2284.758955198" watchObservedRunningTime="2026-07-13 22:37:55.57713225 +0000 UTC m=+2284.759976484" Jul 13 22:37:58.272280 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:37:58.272243 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:01.569339 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:01.569311 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:38:06.791144 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:06.791109 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:06.898753 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:06.898709 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:07.017881 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:07.017835 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:15.461590 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:15.461560 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:15.560370 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:15.560337 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:16.782198 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:16.782164 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:16.885130 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:16.885102 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:17.030713 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.030681 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk"] Jul 13 22:38:17.031213 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.031180 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" podUID="7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46" containerName="maas-api" containerID="cri-o://11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa" gracePeriod=30 Jul 13 22:38:17.056028 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.055992 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:17.284023 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.284000 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:38:17.437300 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.437270 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-openshift-service-ca\") pod \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " Jul 13 22:38:17.437476 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.437318 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-maas-api-tls\") pod \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " Jul 13 22:38:17.437476 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.437416 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4whgj\" (UniqueName: \"kubernetes.io/projected/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-kube-api-access-4whgj\") pod \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\" (UID: \"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46\") " Jul 13 22:38:17.437727 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.437702 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46" (UID: "7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:38:17.439391 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.439365 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46" (UID: "7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:38:17.439512 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.439491 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-kube-api-access-4whgj" (OuterVolumeSpecName: "kube-api-access-4whgj") pod "7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46" (UID: "7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46"). InnerVolumeSpecName "kube-api-access-4whgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:38:17.538259 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.538231 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:38:17.538259 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.538257 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4whgj\" (UniqueName: \"kubernetes.io/projected/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-kube-api-access-4whgj\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:38:17.538259 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.538267 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:38:17.662261 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.662229 2570 generic.go:356] "Generic (PLEG): container finished" podID="7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46" containerID="11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa" exitCode=0 Jul 13 22:38:17.662432 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.662273 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" Jul 13 22:38:17.662432 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.662309 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" event={"ID":"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46","Type":"ContainerDied","Data":"11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa"} Jul 13 22:38:17.662432 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.662359 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk" event={"ID":"7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46","Type":"ContainerDied","Data":"82496e32d5f10d5afc1025e7bfe2c9fb556507e8eeed044c2eeaf1ecc35042bb"} Jul 13 22:38:17.662432 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.662382 2570 scope.go:117] "RemoveContainer" containerID="11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa" Jul 13 22:38:17.671584 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.671567 2570 scope.go:117] "RemoveContainer" containerID="11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa" Jul 13 22:38:17.671894 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:38:17.671842 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa\": container with ID starting with 11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa not found: ID does not exist" containerID="11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa" Jul 13 22:38:17.671953 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.671906 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa"} err="failed to get container status \"11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa\": rpc error: code = NotFound desc = could not find container \"11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa\": container with ID starting with 11eb85f0ea97927f2b0fad185abeb5a420408eb069ce3f3db0ac3b814120c8aa not found: ID does not exist" Jul 13 22:38:17.682126 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.682102 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk"] Jul 13 22:38:17.688388 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:17.688314 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe823c-6b578ddff8-cjkhk"] Jul 13 22:38:19.316400 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:19.316349 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46" path="/var/lib/kubelet/pods/7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46/volumes" Jul 13 22:38:20.839834 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:20.839803 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:20.940728 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:20.940676 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:21.336719 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:21.336688 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:26.180234 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:26.180199 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:27.369902 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:27.369842 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:27.429729 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:27.429702 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:32.061748 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:32.061714 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:32.268740 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:32.268707 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:33.044332 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:33.044303 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:33.155019 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:33.154992 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:42.849531 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:42.849493 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw"] Jul 13 22:38:42.850330 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:42.850272 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" podUID="28add1d8-2469-43a7-9217-961d441f1ddf" containerName="maas-api" containerID="cri-o://25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21" gracePeriod=30 Jul 13 22:38:42.879593 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:42.879568 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:43.099525 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.099503 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:38:43.139763 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.139733 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7l8s\" (UniqueName: \"kubernetes.io/projected/28add1d8-2469-43a7-9217-961d441f1ddf-kube-api-access-m7l8s\") pod \"28add1d8-2469-43a7-9217-961d441f1ddf\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " Jul 13 22:38:43.139947 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.139844 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28add1d8-2469-43a7-9217-961d441f1ddf-openshift-service-ca\") pod \"28add1d8-2469-43a7-9217-961d441f1ddf\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " Jul 13 22:38:43.140016 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.139972 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28add1d8-2469-43a7-9217-961d441f1ddf-maas-api-tls\") pod \"28add1d8-2469-43a7-9217-961d441f1ddf\" (UID: \"28add1d8-2469-43a7-9217-961d441f1ddf\") " Jul 13 22:38:43.140225 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.140201 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28add1d8-2469-43a7-9217-961d441f1ddf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28add1d8-2469-43a7-9217-961d441f1ddf" (UID: "28add1d8-2469-43a7-9217-961d441f1ddf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:38:43.141835 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.141811 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28add1d8-2469-43a7-9217-961d441f1ddf-kube-api-access-m7l8s" (OuterVolumeSpecName: "kube-api-access-m7l8s") pod "28add1d8-2469-43a7-9217-961d441f1ddf" (UID: "28add1d8-2469-43a7-9217-961d441f1ddf"). InnerVolumeSpecName "kube-api-access-m7l8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:38:43.141981 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.141959 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28add1d8-2469-43a7-9217-961d441f1ddf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28add1d8-2469-43a7-9217-961d441f1ddf" (UID: "28add1d8-2469-43a7-9217-961d441f1ddf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:38:43.240626 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.240598 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28add1d8-2469-43a7-9217-961d441f1ddf-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:38:43.240626 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.240625 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7l8s\" (UniqueName: \"kubernetes.io/projected/28add1d8-2469-43a7-9217-961d441f1ddf-kube-api-access-m7l8s\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:38:43.240805 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.240637 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28add1d8-2469-43a7-9217-961d441f1ddf-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:38:43.783022 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.782989 2570 generic.go:356] "Generic (PLEG): container finished" podID="28add1d8-2469-43a7-9217-961d441f1ddf" containerID="25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21" exitCode=0 Jul 13 22:38:43.783199 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.783036 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" Jul 13 22:38:43.783199 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.783056 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" event={"ID":"28add1d8-2469-43a7-9217-961d441f1ddf","Type":"ContainerDied","Data":"25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21"} Jul 13 22:38:43.783199 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.783100 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw" event={"ID":"28add1d8-2469-43a7-9217-961d441f1ddf","Type":"ContainerDied","Data":"d8f470659246877babc1fe32642f3eab663e6d5cb7fc2a1073b503b9556bdc0d"} Jul 13 22:38:43.783199 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.783121 2570 scope.go:117] "RemoveContainer" containerID="25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21" Jul 13 22:38:43.791346 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.791330 2570 scope.go:117] "RemoveContainer" containerID="25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21" Jul 13 22:38:43.791588 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:38:43.791567 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21\": container with ID starting with 25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21 not found: ID does not exist" containerID="25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21" Jul 13 22:38:43.791645 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.791596 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21"} err="failed to get container status \"25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21\": rpc error: code = NotFound desc = could not find container \"25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21\": container with ID starting with 25d991e91da40e7ca9958d61ff35d7cc0b89748baaa6c44723e9d2672ca79d21 not found: ID does not exist" Jul 13 22:38:43.797590 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.797571 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw"] Jul 13 22:38:43.803407 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:43.803389 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2a08d3-8597c496dd-42jsw"] Jul 13 22:38:45.316842 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:45.316809 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28add1d8-2469-43a7-9217-961d441f1ddf" path="/var/lib/kubelet/pods/28add1d8-2469-43a7-9217-961d441f1ddf/volumes" Jul 13 22:38:47.482731 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:47.482689 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:53.525576 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:53.525546 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:53.593143 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:53.593111 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:58.146029 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:58.145999 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:58.366387 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:58.366356 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:58.692750 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:58.692718 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:59.006989 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:59.006914 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:38:59.370833 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:38:59.370809 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:02.242766 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.242728 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:02.346829 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.346799 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:02.488899 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.488869 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6"] Jul 13 22:39:02.489312 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.489296 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28add1d8-2469-43a7-9217-961d441f1ddf" containerName="maas-api" Jul 13 22:39:02.489403 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.489315 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="28add1d8-2469-43a7-9217-961d441f1ddf" containerName="maas-api" Jul 13 22:39:02.489403 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.489329 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46" containerName="maas-api" Jul 13 22:39:02.489403 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.489337 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46" containerName="maas-api" Jul 13 22:39:02.489551 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.489409 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f5afa9a-9fc3-4d2c-8345-f3d1f1a07a46" containerName="maas-api" Jul 13 22:39:02.489551 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.489421 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="28add1d8-2469-43a7-9217-961d441f1ddf" containerName="maas-api" Jul 13 22:39:02.492628 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.492608 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:02.495200 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.495146 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-13bc0c-serving-cert\"" Jul 13 22:39:02.503570 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.503548 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6"] Jul 13 22:39:02.546845 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.546818 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:02.598026 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.597985 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e338df-d3b9-4cb5-a534-d76b68a986c5-maas-api-tls\") pod \"maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:02.598171 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.598032 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6657\" (UniqueName: \"kubernetes.io/projected/72e338df-d3b9-4cb5-a534-d76b68a986c5-kube-api-access-d6657\") pod \"maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:02.598171 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.598140 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e338df-d3b9-4cb5-a534-d76b68a986c5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:02.666208 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.666174 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:02.699353 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.699318 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e338df-d3b9-4cb5-a534-d76b68a986c5-maas-api-tls\") pod \"maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:02.699454 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.699369 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6657\" (UniqueName: \"kubernetes.io/projected/72e338df-d3b9-4cb5-a534-d76b68a986c5-kube-api-access-d6657\") pod \"maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:02.699454 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.699406 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e338df-d3b9-4cb5-a534-d76b68a986c5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:02.699965 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.699947 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e338df-d3b9-4cb5-a534-d76b68a986c5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:02.701685 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.701663 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e338df-d3b9-4cb5-a534-d76b68a986c5-maas-api-tls\") pod \"maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:02.706889 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.706865 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6657\" (UniqueName: \"kubernetes.io/projected/72e338df-d3b9-4cb5-a534-d76b68a986c5-kube-api-access-d6657\") pod \"maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:02.805955 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.805874 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:02.935722 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:02.935690 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6"] Jul 13 22:39:02.937527 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:39:02.937502 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e338df_d3b9_4cb5_a534_d76b68a986c5.slice/crio-9f403cd3e4919ba4c0446cbbfae7be2e70eed53f606d1e64fc635095253e059e WatchSource:0}: Error finding container 9f403cd3e4919ba4c0446cbbfae7be2e70eed53f606d1e64fc635095253e059e: Status 404 returned error can't find the container with id 9f403cd3e4919ba4c0446cbbfae7be2e70eed53f606d1e64fc635095253e059e Jul 13 22:39:03.026840 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:03.026806 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:03.128698 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:03.128670 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:03.870673 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:03.870639 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" event={"ID":"72e338df-d3b9-4cb5-a534-d76b68a986c5","Type":"ContainerStarted","Data":"c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b"} Jul 13 22:39:03.871081 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:03.870689 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" event={"ID":"72e338df-d3b9-4cb5-a534-d76b68a986c5","Type":"ContainerStarted","Data":"9f403cd3e4919ba4c0446cbbfae7be2e70eed53f606d1e64fc635095253e059e"} Jul 13 22:39:03.871081 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:03.870746 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:03.891231 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:03.891195 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" podStartSLOduration=1.526067844 podStartE2EDuration="1.891181802s" totalImagesPullingTime="365.113958ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:39:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:39:03.890345446 +0000 UTC m=+2353.073189692" watchObservedRunningTime="2026-07-13 22:39:03.891181802 +0000 UTC m=+2353.074026031" Jul 13 22:39:06.737402 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:06.737363 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:07.052878 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:07.052769 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:07.366770 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:07.366699 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:07.602804 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:07.602773 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:09.883312 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:09.883280 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:39:10.315621 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.315514 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:10.398089 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.398057 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:10.398533 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.398513 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt"] Jul 13 22:39:10.401985 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.401968 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:10.404331 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.404314 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-bdc0cb-serving-cert\"" Jul 13 22:39:10.417992 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.417968 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt"] Jul 13 22:39:10.465570 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.465538 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:10.465727 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.465610 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-maas-api-tls\") pod \"maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:10.465727 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.465669 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xws9h\" (UniqueName: \"kubernetes.io/projected/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-kube-api-access-xws9h\") pod \"maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:10.508592 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.508555 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:10.566634 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.566563 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xws9h\" (UniqueName: \"kubernetes.io/projected/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-kube-api-access-xws9h\") pod \"maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:10.566634 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.566622 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:10.566816 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.566652 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-maas-api-tls\") pod \"maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:10.566816 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:39:10.566746 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-serving-cert: secret "maas-api-e2e-shared-b-bdc0cb-serving-cert" not found Jul 13 22:39:10.566816 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:39:10.566798 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-maas-api-tls podName:4b15880f-ada5-4c3c-bfd2-ddab40473d9d nodeName:}" failed. No retries permitted until 2026-07-13 22:39:11.06678205 +0000 UTC m=+2360.249626258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-maas-api-tls") pod "maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" (UID: "4b15880f-ada5-4c3c-bfd2-ddab40473d9d") : secret "maas-api-e2e-shared-b-bdc0cb-serving-cert" not found Jul 13 22:39:10.567275 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.567258 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:10.574737 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.574706 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xws9h\" (UniqueName: \"kubernetes.io/projected/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-kube-api-access-xws9h\") pod \"maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:10.904483 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:10.904449 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:11.003129 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:11.003101 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:11.071197 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:11.071166 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-maas-api-tls\") pod \"maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:11.073639 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:11.073612 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-maas-api-tls\") pod \"maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:11.103439 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:11.103409 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:11.311762 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:11.311684 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:11.441114 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:11.441093 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt"] Jul 13 22:39:11.442911 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:39:11.442883 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b15880f_ada5_4c3c_bfd2_ddab40473d9d.slice/crio-1d75320a2a39f7021d25aabdd61904cec2c61165fc6c64f3155a86f4b7f23c59 WatchSource:0}: Error finding container 1d75320a2a39f7021d25aabdd61904cec2c61165fc6c64f3155a86f4b7f23c59: Status 404 returned error can't find the container with id 1d75320a2a39f7021d25aabdd61904cec2c61165fc6c64f3155a86f4b7f23c59 Jul 13 22:39:11.915122 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:11.915097 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" event={"ID":"4b15880f-ada5-4c3c-bfd2-ddab40473d9d","Type":"ContainerStarted","Data":"1d75320a2a39f7021d25aabdd61904cec2c61165fc6c64f3155a86f4b7f23c59"} Jul 13 22:39:12.920200 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:12.920165 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" event={"ID":"4b15880f-ada5-4c3c-bfd2-ddab40473d9d","Type":"ContainerStarted","Data":"1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31"} Jul 13 22:39:12.920639 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:12.920295 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:12.938557 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:12.938503 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" podStartSLOduration=2.504568007 podStartE2EDuration="2.938488008s" totalImagesPullingTime="433.920001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:39:12.937422593 +0000 UTC m=+2362.120266823" watchObservedRunningTime="2026-07-13 22:39:12.938488008 +0000 UTC m=+2362.121332237" Jul 13 22:39:14.785128 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:14.785094 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:18.930239 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:18.930206 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:39:23.937350 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:23.937325 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:24.036146 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:24.036106 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:29.495883 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:29.495814 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:32.716707 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:32.716671 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:32.815462 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:32.815432 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:39:51.369455 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:51.369421 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:39:51.389263 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:51.389237 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:39:59.988933 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:39:59.988902 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:42:05.883694 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:42:05.883650 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:42:07.092718 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:42:07.092671 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:42:13.472084 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:42:13.472056 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:42:51.853025 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:42:51.852987 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:42:51.952557 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:42:51.952524 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:42:52.648079 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:42:52.648050 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:42:54.884311 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:42:54.884273 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:43:00.179056 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:00.179025 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:43:41.892797 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:41.892764 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:43:42.856648 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:42.856617 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:43:46.647740 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.647702 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9"] Jul 13 22:43:46.651163 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.651146 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.653516 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.653490 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-13bc0c\"/\"rate-test-model-13bc0c-kserve-self-signed-certs\"" Jul 13 22:43:46.653620 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.653514 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-13bc0c\"/\"kube-root-ca.crt\"" Jul 13 22:43:46.653620 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.653563 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-13bc0c\"/\"openshift-service-ca.crt\"" Jul 13 22:43:46.653620 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.653517 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-13bc0c\"/\"default-dockercfg-58ljc\"" Jul 13 22:43:46.659069 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.659048 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9"] Jul 13 22:43:46.754635 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.753040 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tmp-dir\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.754635 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.753144 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-home\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.754635 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.753179 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tls-certs\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.754635 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.753207 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqh8g\" (UniqueName: \"kubernetes.io/projected/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-kube-api-access-nqh8g\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.754635 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.753241 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-dshm\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.754635 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.753262 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-model-cache\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.755368 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.755336 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:43:46.854495 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.854457 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tmp-dir\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.854671 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.854524 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-home\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.854671 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.854544 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tls-certs\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.854671 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.854561 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqh8g\" (UniqueName: \"kubernetes.io/projected/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-kube-api-access-nqh8g\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.854671 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.854584 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-dshm\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.854671 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.854599 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-model-cache\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.854992 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.854957 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tmp-dir\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.855037 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.854986 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-home\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.855037 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.855009 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-model-cache\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.856881 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.856841 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-dshm\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.857041 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.857025 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tls-certs\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.861691 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.861674 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqh8g\" (UniqueName: \"kubernetes.io/projected/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-kube-api-access-nqh8g\") pod \"rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:46.960921 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:46.960826 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:47.095553 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:47.095531 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9"] Jul 13 22:43:47.097963 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:43:47.097934 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6b97f8b_d2d4_4013_97ff_4f92f1c0267e.slice/crio-ceb0a576bc895af46f1cd29dea2049ae0178364dcfba6d6c89dc411068700a78 WatchSource:0}: Error finding container ceb0a576bc895af46f1cd29dea2049ae0178364dcfba6d6c89dc411068700a78: Status 404 returned error can't find the container with id ceb0a576bc895af46f1cd29dea2049ae0178364dcfba6d6c89dc411068700a78 Jul 13 22:43:47.176338 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:47.176301 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" event={"ID":"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e","Type":"ContainerStarted","Data":"240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6"} Jul 13 22:43:47.176461 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:47.176368 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" event={"ID":"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e","Type":"ContainerStarted","Data":"ceb0a576bc895af46f1cd29dea2049ae0178364dcfba6d6c89dc411068700a78"} Jul 13 22:43:47.193116 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:47.193066 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" podStartSLOduration=1.193048048 podStartE2EDuration="1.193048048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:43:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:43:47.192524384 +0000 UTC m=+2636.375368636" watchObservedRunningTime="2026-07-13 22:43:47.193048048 +0000 UTC m=+2636.375892280" Jul 13 22:43:56.961887 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:56.961844 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:56.962293 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:56.961907 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:56.974189 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:56.974167 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:43:57.238240 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:43:57.238170 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:44:07.760078 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:44:07.760041 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:44:24.698243 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:44:24.698200 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:44:24.802879 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:44:24.802836 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:44:33.635194 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:44:33.635160 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:44:33.736243 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:44:33.736214 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:44:51.405981 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:44:51.405801 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:44:51.420389 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:44:51.420356 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9grcb_099fa6a3-1b5f-4f2f-b1e3-e72d817dbec8/ovn-acl-logging/0.log" Jul 13 22:44:56.571660 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:44:56.571624 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9"] Jul 13 22:44:56.591217 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:44:56.591180 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:44:57.447419 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:44:57.447392 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:00.143388 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:00.143358 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv"] Jul 13 22:45:00.146587 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:00.146572 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv" Jul 13 22:45:00.148604 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:00.148580 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k5z5d\"" Jul 13 22:45:00.157371 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:00.157344 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv"] Jul 13 22:45:00.158745 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:00.158728 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv" Jul 13 22:45:00.284529 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:00.284508 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv"] Jul 13 22:45:00.286505 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:45:00.286478 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc01752_43cf_434c_abf3_7e68db5107ea.slice/crio-36be04c1eab66472111023beaabc0b7d534ee1f93dd831b9684bd61edd85fffb WatchSource:0}: Error finding container 36be04c1eab66472111023beaabc0b7d534ee1f93dd831b9684bd61edd85fffb: Status 404 returned error can't find the container with id 36be04c1eab66472111023beaabc0b7d534ee1f93dd831b9684bd61edd85fffb Jul 13 22:45:00.288197 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:00.288183 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 22:45:00.509888 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:00.509786 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv" event={"ID":"dbc01752-43cf-434c-abf3-7e68db5107ea","Type":"ContainerStarted","Data":"36be04c1eab66472111023beaabc0b7d534ee1f93dd831b9684bd61edd85fffb"} Jul 13 22:45:03.526905 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:03.526868 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv" event={"ID":"dbc01752-43cf-434c-abf3-7e68db5107ea","Type":"ContainerStarted","Data":"810ce93ba1f16c75af60a4b2a2f625e97427c72705a5de08aff87416aaa140bf"} Jul 13 22:45:03.543298 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:03.543237 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv" podStartSLOduration=1.15176652 podStartE2EDuration="3.543221029s" totalImagesPullingTime="2.391454509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:45:03.542174903 +0000 UTC m=+2712.725019134" watchObservedRunningTime="2026-07-13 22:45:03.543221029 +0000 UTC m=+2712.726065262" Jul 13 22:45:04.532918 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:04.532886 2570 generic.go:356] "Generic (PLEG): container finished" podID="dbc01752-43cf-434c-abf3-7e68db5107ea" containerID="810ce93ba1f16c75af60a4b2a2f625e97427c72705a5de08aff87416aaa140bf" exitCode=0 Jul 13 22:45:04.533354 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:04.532955 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv" event={"ID":"dbc01752-43cf-434c-abf3-7e68db5107ea","Type":"ContainerDied","Data":"810ce93ba1f16c75af60a4b2a2f625e97427c72705a5de08aff87416aaa140bf"} Jul 13 22:45:05.667441 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:05.667416 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv" Jul 13 22:45:06.544259 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:06.544229 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv" Jul 13 22:45:06.544441 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:06.544277 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv" event={"ID":"dbc01752-43cf-434c-abf3-7e68db5107ea","Type":"ContainerDied","Data":"36be04c1eab66472111023beaabc0b7d534ee1f93dd831b9684bd61edd85fffb"} Jul 13 22:45:06.544441 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:06.544323 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36be04c1eab66472111023beaabc0b7d534ee1f93dd831b9684bd61edd85fffb" Jul 13 22:45:11.593338 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.593282 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" podUID="c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" containerName="main" containerID="cri-o://240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6" gracePeriod=45 Jul 13 22:45:11.847956 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.847905 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:45:11.980191 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.980162 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqh8g\" (UniqueName: \"kubernetes.io/projected/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-kube-api-access-nqh8g\") pod \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " Jul 13 22:45:11.980323 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.980203 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tmp-dir\") pod \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " Jul 13 22:45:11.980323 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.980230 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tls-certs\") pod \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " Jul 13 22:45:11.980323 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.980248 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-home\") pod \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " Jul 13 22:45:11.980323 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.980278 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-model-cache\") pod \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " Jul 13 22:45:11.980323 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.980297 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-dshm\") pod \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\" (UID: \"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e\") " Jul 13 22:45:11.980595 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.980493 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" (UID: "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:45:11.980595 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.980528 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-home" (OuterVolumeSpecName: "home") pod "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" (UID: "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:45:11.980692 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.980596 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-model-cache" (OuterVolumeSpecName: "model-cache") pod "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" (UID: "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:45:11.982410 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.982378 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-dshm" (OuterVolumeSpecName: "dshm") pod "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" (UID: "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 22:45:11.982519 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.982441 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" (UID: "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:45:11.982519 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:11.982460 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-kube-api-access-nqh8g" (OuterVolumeSpecName: "kube-api-access-nqh8g") pod "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" (UID: "c6b97f8b-d2d4-4013-97ff-4f92f1c0267e"). InnerVolumeSpecName "kube-api-access-nqh8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:45:12.081126 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.081099 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-model-cache\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:12.081126 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.081121 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-dshm\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:12.081126 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.081130 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqh8g\" (UniqueName: \"kubernetes.io/projected/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-kube-api-access-nqh8g\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:12.081332 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.081160 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tmp-dir\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:12.081332 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.081169 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-tls-certs\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:12.081332 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.081178 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e-home\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:12.574254 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.574225 2570 generic.go:356] "Generic (PLEG): container finished" podID="c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" containerID="240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6" exitCode=0 Jul 13 22:45:12.574454 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.574265 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" Jul 13 22:45:12.574454 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.574313 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" event={"ID":"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e","Type":"ContainerDied","Data":"240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6"} Jul 13 22:45:12.574454 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.574358 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9" event={"ID":"c6b97f8b-d2d4-4013-97ff-4f92f1c0267e","Type":"ContainerDied","Data":"ceb0a576bc895af46f1cd29dea2049ae0178364dcfba6d6c89dc411068700a78"} Jul 13 22:45:12.574454 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.574382 2570 scope.go:117] "RemoveContainer" containerID="240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6" Jul 13 22:45:12.583299 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.583275 2570 scope.go:117] "RemoveContainer" containerID="240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6" Jul 13 22:45:12.583531 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:45:12.583513 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6\": container with ID starting with 240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6 not found: ID does not exist" containerID="240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6" Jul 13 22:45:12.583587 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.583539 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6"} err="failed to get container status \"240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6\": rpc error: code = NotFound desc = could not find container \"240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6\": container with ID starting with 240723705fd4ea0e107e6f00bddd3c223d001579c28f2575680e045061a374f6 not found: ID does not exist" Jul 13 22:45:12.594132 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.594108 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9"] Jul 13 22:45:12.597412 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:12.597391 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-13bc0c/rate-test-model-13bc0c-kserve-7b69d67df9-9f9q9"] Jul 13 22:45:13.317114 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:13.317074 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" path="/var/lib/kubelet/pods/c6b97f8b-d2d4-4013-97ff-4f92f1c0267e/volumes" Jul 13 22:45:17.562570 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:17.562544 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:17.666519 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:17.666486 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:27.500912 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:27.500880 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6"] Jul 13 22:45:27.501402 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:27.501113 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" podUID="72e338df-d3b9-4cb5-a534-d76b68a986c5" containerName="maas-api" containerID="cri-o://c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b" gracePeriod=30 Jul 13 22:45:27.521722 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:27.521691 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:27.528966 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:27.528944 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv"] Jul 13 22:45:27.531619 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:27.531599 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-13bc0c-29733045-64wnv"] Jul 13 22:45:27.950324 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:27.950302 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:45:28.123018 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.122997 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e338df-d3b9-4cb5-a534-d76b68a986c5-maas-api-tls\") pod \"72e338df-d3b9-4cb5-a534-d76b68a986c5\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " Jul 13 22:45:28.123177 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.123029 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6657\" (UniqueName: \"kubernetes.io/projected/72e338df-d3b9-4cb5-a534-d76b68a986c5-kube-api-access-d6657\") pod \"72e338df-d3b9-4cb5-a534-d76b68a986c5\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " Jul 13 22:45:28.123177 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.123063 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e338df-d3b9-4cb5-a534-d76b68a986c5-openshift-service-ca\") pod \"72e338df-d3b9-4cb5-a534-d76b68a986c5\" (UID: \"72e338df-d3b9-4cb5-a534-d76b68a986c5\") " Jul 13 22:45:28.123483 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.123458 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72e338df-d3b9-4cb5-a534-d76b68a986c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "72e338df-d3b9-4cb5-a534-d76b68a986c5" (UID: "72e338df-d3b9-4cb5-a534-d76b68a986c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:45:28.125039 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.125012 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72e338df-d3b9-4cb5-a534-d76b68a986c5-kube-api-access-d6657" (OuterVolumeSpecName: "kube-api-access-d6657") pod "72e338df-d3b9-4cb5-a534-d76b68a986c5" (UID: "72e338df-d3b9-4cb5-a534-d76b68a986c5"). InnerVolumeSpecName "kube-api-access-d6657". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:45:28.125135 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.125047 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72e338df-d3b9-4cb5-a534-d76b68a986c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "72e338df-d3b9-4cb5-a534-d76b68a986c5" (UID: "72e338df-d3b9-4cb5-a534-d76b68a986c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:45:28.224281 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.224251 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e338df-d3b9-4cb5-a534-d76b68a986c5-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:28.224281 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.224276 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6657\" (UniqueName: \"kubernetes.io/projected/72e338df-d3b9-4cb5-a534-d76b68a986c5-kube-api-access-d6657\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:28.224281 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.224287 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e338df-d3b9-4cb5-a534-d76b68a986c5-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:28.647434 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.647401 2570 generic.go:356] "Generic (PLEG): container finished" podID="72e338df-d3b9-4cb5-a534-d76b68a986c5" containerID="c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b" exitCode=0 Jul 13 22:45:28.647970 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.647446 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" Jul 13 22:45:28.647970 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.647479 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" event={"ID":"72e338df-d3b9-4cb5-a534-d76b68a986c5","Type":"ContainerDied","Data":"c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b"} Jul 13 22:45:28.647970 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.647532 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6" event={"ID":"72e338df-d3b9-4cb5-a534-d76b68a986c5","Type":"ContainerDied","Data":"9f403cd3e4919ba4c0446cbbfae7be2e70eed53f606d1e64fc635095253e059e"} Jul 13 22:45:28.647970 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.647554 2570 scope.go:117] "RemoveContainer" containerID="c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b" Jul 13 22:45:28.656128 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.656041 2570 scope.go:117] "RemoveContainer" containerID="c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b" Jul 13 22:45:28.656299 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:45:28.656279 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b\": container with ID starting with c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b not found: ID does not exist" containerID="c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b" Jul 13 22:45:28.656376 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.656312 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b"} err="failed to get container status \"c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b\": rpc error: code = NotFound desc = could not find container \"c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b\": container with ID starting with c67d6d367bbeee5e38ff5d0e6a956bfdcb96d91450ec295950ddcc3b8aa8ca9b not found: ID does not exist" Jul 13 22:45:28.667320 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.667285 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6"] Jul 13 22:45:28.668924 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:28.668903 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-13bc0c-74b5c78786-mv2k6"] Jul 13 22:45:29.316188 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:29.316152 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72e338df-d3b9-4cb5-a534-d76b68a986c5" path="/var/lib/kubelet/pods/72e338df-d3b9-4cb5-a534-d76b68a986c5/volumes" Jul 13 22:45:29.316459 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:29.316443 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbc01752-43cf-434c-abf3-7e68db5107ea" path="/var/lib/kubelet/pods/dbc01752-43cf-434c-abf3-7e68db5107ea/volumes" Jul 13 22:45:32.157587 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:32.157556 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:38.123750 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:38.123714 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:38.222443 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:38.222413 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:42.809739 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:42.809709 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:43.035278 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.035243 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:43.153485 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.153454 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v"] Jul 13 22:45:43.153814 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.153803 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbc01752-43cf-434c-abf3-7e68db5107ea" containerName="cleanup" Jul 13 22:45:43.153882 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.153816 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbc01752-43cf-434c-abf3-7e68db5107ea" containerName="cleanup" Jul 13 22:45:43.153882 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.153836 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" containerName="main" Jul 13 22:45:43.153882 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.153842 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" containerName="main" Jul 13 22:45:43.153882 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.153862 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72e338df-d3b9-4cb5-a534-d76b68a986c5" containerName="maas-api" Jul 13 22:45:43.153882 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.153869 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="72e338df-d3b9-4cb5-a534-d76b68a986c5" containerName="maas-api" Jul 13 22:45:43.154039 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.153920 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="72e338df-d3b9-4cb5-a534-d76b68a986c5" containerName="maas-api" Jul 13 22:45:43.154039 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.153930 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6b97f8b-d2d4-4013-97ff-4f92f1c0267e" containerName="main" Jul 13 22:45:43.154039 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.153938 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbc01752-43cf-434c-abf3-7e68db5107ea" containerName="cleanup" Jul 13 22:45:43.156721 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.156706 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v" Jul 13 22:45:43.158561 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.158532 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k5z5d\"" Jul 13 22:45:43.162479 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.162457 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v"] Jul 13 22:45:43.235735 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.235705 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8472184c-0683-4c68-a8b7-65c81ad2043a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v\" (UID: \"8472184c-0683-4c68-a8b7-65c81ad2043a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v" Jul 13 22:45:43.336654 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.336621 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8472184c-0683-4c68-a8b7-65c81ad2043a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v\" (UID: \"8472184c-0683-4c68-a8b7-65c81ad2043a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v" Jul 13 22:45:43.337205 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.337187 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8472184c-0683-4c68-a8b7-65c81ad2043a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v\" (UID: \"8472184c-0683-4c68-a8b7-65c81ad2043a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v" Jul 13 22:45:43.468263 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.468179 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v" Jul 13 22:45:43.598434 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.598402 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v"] Jul 13 22:45:43.601363 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:45:43.601341 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8472184c_0683_4c68_a8b7_65c81ad2043a.slice/crio-f66a149ef10a05d9950822549bb20f4ed6cdd1c3654497fc5535e5414908bf87 WatchSource:0}: Error finding container f66a149ef10a05d9950822549bb20f4ed6cdd1c3654497fc5535e5414908bf87: Status 404 returned error can't find the container with id f66a149ef10a05d9950822549bb20f4ed6cdd1c3654497fc5535e5414908bf87 Jul 13 22:45:43.717797 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.717761 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v" event={"ID":"8472184c-0683-4c68-a8b7-65c81ad2043a","Type":"ContainerStarted","Data":"f66a149ef10a05d9950822549bb20f4ed6cdd1c3654497fc5535e5414908bf87"} Jul 13 22:45:43.837316 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.837251 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:43.936065 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:43.936039 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:44.724382 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:44.724294 2570 generic.go:356] "Generic (PLEG): container finished" podID="8472184c-0683-4c68-a8b7-65c81ad2043a" containerID="45ecef5d52e560c6faed1abd01a858b7f6e232ab342c153a58e9beada2221251" exitCode=0 Jul 13 22:45:44.724382 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:44.724370 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v" event={"ID":"8472184c-0683-4c68-a8b7-65c81ad2043a","Type":"ContainerDied","Data":"45ecef5d52e560c6faed1abd01a858b7f6e232ab342c153a58e9beada2221251"} Jul 13 22:45:45.866310 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:45.866280 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v" Jul 13 22:45:45.962138 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:45.962098 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8472184c-0683-4c68-a8b7-65c81ad2043a-maas-api-service-ca\") pod \"8472184c-0683-4c68-a8b7-65c81ad2043a\" (UID: \"8472184c-0683-4c68-a8b7-65c81ad2043a\") " Jul 13 22:45:45.962451 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:45.962425 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8472184c-0683-4c68-a8b7-65c81ad2043a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8472184c-0683-4c68-a8b7-65c81ad2043a" (UID: "8472184c-0683-4c68-a8b7-65c81ad2043a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:45:46.062880 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:46.062782 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8472184c-0683-4c68-a8b7-65c81ad2043a-maas-api-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:46.735730 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:46.735699 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v" Jul 13 22:45:46.735730 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:46.735716 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v" event={"ID":"8472184c-0683-4c68-a8b7-65c81ad2043a","Type":"ContainerDied","Data":"f66a149ef10a05d9950822549bb20f4ed6cdd1c3654497fc5535e5414908bf87"} Jul 13 22:45:46.735957 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:46.735755 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f66a149ef10a05d9950822549bb20f4ed6cdd1c3654497fc5535e5414908bf87" Jul 13 22:45:53.619603 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:53.619566 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt"] Jul 13 22:45:53.620041 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:53.619808 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" podUID="4b15880f-ada5-4c3c-bfd2-ddab40473d9d" containerName="maas-api" containerID="cri-o://1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31" gracePeriod=30 Jul 13 22:45:53.655944 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:53.655911 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:45:53.925775 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:53.925742 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" podUID="4b15880f-ada5-4c3c-bfd2-ddab40473d9d" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.49:8443/health\": dial tcp 10.133.0.49:8443: connect: connection refused" Jul 13 22:45:54.064071 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.064050 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:45:54.117919 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.117893 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-openshift-service-ca\") pod \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " Jul 13 22:45:54.118085 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.117990 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xws9h\" (UniqueName: \"kubernetes.io/projected/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-kube-api-access-xws9h\") pod \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " Jul 13 22:45:54.118085 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.118021 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-maas-api-tls\") pod \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\" (UID: \"4b15880f-ada5-4c3c-bfd2-ddab40473d9d\") " Jul 13 22:45:54.118247 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.118219 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b15880f-ada5-4c3c-bfd2-ddab40473d9d" (UID: "4b15880f-ada5-4c3c-bfd2-ddab40473d9d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:45:54.120010 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.119978 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b15880f-ada5-4c3c-bfd2-ddab40473d9d" (UID: "4b15880f-ada5-4c3c-bfd2-ddab40473d9d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 22:45:54.120115 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.120086 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-kube-api-access-xws9h" (OuterVolumeSpecName: "kube-api-access-xws9h") pod "4b15880f-ada5-4c3c-bfd2-ddab40473d9d" (UID: "4b15880f-ada5-4c3c-bfd2-ddab40473d9d"). InnerVolumeSpecName "kube-api-access-xws9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 22:45:54.218707 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.218638 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xws9h\" (UniqueName: \"kubernetes.io/projected/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-kube-api-access-xws9h\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:54.218707 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.218664 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-maas-api-tls\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:54.218707 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.218674 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b15880f-ada5-4c3c-bfd2-ddab40473d9d-openshift-service-ca\") on node \"ip-10-0-143-97.ec2.internal\" DevicePath \"\"" Jul 13 22:45:54.775618 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.775585 2570 generic.go:356] "Generic (PLEG): container finished" podID="4b15880f-ada5-4c3c-bfd2-ddab40473d9d" containerID="1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31" exitCode=0 Jul 13 22:45:54.776024 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.775627 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" Jul 13 22:45:54.776024 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.775661 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" event={"ID":"4b15880f-ada5-4c3c-bfd2-ddab40473d9d","Type":"ContainerDied","Data":"1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31"} Jul 13 22:45:54.776024 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.775717 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt" event={"ID":"4b15880f-ada5-4c3c-bfd2-ddab40473d9d","Type":"ContainerDied","Data":"1d75320a2a39f7021d25aabdd61904cec2c61165fc6c64f3155a86f4b7f23c59"} Jul 13 22:45:54.776024 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.775740 2570 scope.go:117] "RemoveContainer" containerID="1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31" Jul 13 22:45:54.784202 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.784187 2570 scope.go:117] "RemoveContainer" containerID="1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31" Jul 13 22:45:54.784454 ip-10-0-143-97 kubenswrapper[2570]: E0713 22:45:54.784433 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31\": container with ID starting with 1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31 not found: ID does not exist" containerID="1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31" Jul 13 22:45:54.784504 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.784462 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31"} err="failed to get container status \"1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31\": rpc error: code = NotFound desc = could not find container \"1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31\": container with ID starting with 1604dc6d8b9e63f043ab0b6ff080f7804ae4193ca864003dfb57bd75d51a0f31 not found: ID does not exist" Jul 13 22:45:54.796040 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.796021 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt"] Jul 13 22:45:54.799874 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:54.799838 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc0cb-7cb69f97d7-g55mt"] Jul 13 22:45:55.316544 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:55.316512 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b15880f-ada5-4c3c-bfd2-ddab40473d9d" path="/var/lib/kubelet/pods/4b15880f-ada5-4c3c-bfd2-ddab40473d9d/volumes" Jul 13 22:45:58.249732 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:45:58.249696 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:46:04.107391 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:04.107360 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:46:04.187310 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:04.187279 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:46:05.119570 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:05.119535 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:46:08.901712 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:08.901676 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:46:09.133279 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:09.133233 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:46:14.415025 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:14.414945 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6dc46d8554-mq6tm_3944875c-dd62-4f5e-9e75-c3cf8dbe69bc/manager/0.log" Jul 13 22:46:14.998870 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:14.998822 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67ddb65fc-vn66g_05edc547-1871-4cdc-8770-d18c09e2dba2/copy-manifests/0.log" Jul 13 22:46:15.015450 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:15.015422 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67ddb65fc-vn66g_05edc547-1871-4cdc-8770-d18c09e2dba2/manager/0.log" Jul 13 22:46:15.367290 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:15.367260 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6454d8c544-jcfgz_5ad98b2f-3bf7-4019-9076-361c089c95b5/maas-api/0.log" Jul 13 22:46:17.006067 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:17.006038 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-bdc0cb-jwr2v_8472184c-0683-4c68-a8b7-65c81ad2043a/revoke-keys/0.log" Jul 13 22:46:17.109785 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:17.109759 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8khfd_10d24f67-b1cc-4b6b-a6d0-320b0f4b957a/postgres/0.log" Jul 13 22:46:19.625472 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:19.625440 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-d6ljs_3071b09b-3802-4a16-b299-2e04458b8ecd/discovery/0.log" Jul 13 22:46:20.074796 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:20.074720 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-8q5b7_05c0b422-638c-4b6f-a42e-c75fe345e9d7/payload-pre-processing/0.log" Jul 13 22:46:20.191052 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:20.191022 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-5czcx_a0e46651-ae37-4f23-862d-a628630d6e40/payload-processing/0.log" Jul 13 22:46:20.856108 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:20.856080 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vnbd4_b5a56eb6-d495-4d64-a600-7cd96c8b62c9/main/0.log" Jul 13 22:46:20.965639 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:20.965617 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwpwps_539748de-2362-4ece-834a-47d4af878dec/main/0.log" Jul 13 22:46:24.141453 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:24.141406 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:46:25.429425 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:25.429376 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:46:25.587972 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:25.587940 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" limit=1024 Jul 13 22:46:28.275818 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:28.275786 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4w92z_a11f9f29-be8e-4926-af3d-f2d4ed6b00d6/global-pull-secret-syncer/0.log" Jul 13 22:46:28.398495 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:28.398468 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nfsn4_70c21dbf-84f1-49fa-8c8d-a885d451262a/konnectivity-agent/0.log" Jul 13 22:46:28.529897 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:28.529801 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-97.ec2.internal_fa7dc64b698a22b7103fb912a9c68e2a/haproxy/0.log" Jul 13 22:46:36.586763 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:36.586732 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-jsfmg_e0ff5faa-084b-44a6-84d7-bc5e25f1b41c/kube-state-metrics/0.log" Jul 13 22:46:36.604550 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:36.604512 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-jsfmg_e0ff5faa-084b-44a6-84d7-bc5e25f1b41c/kube-rbac-proxy-main/0.log" Jul 13 22:46:36.623468 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:36.623443 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-jsfmg_e0ff5faa-084b-44a6-84d7-bc5e25f1b41c/kube-rbac-proxy-self/0.log" Jul 13 22:46:36.789278 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:36.789252 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7s9sr_5c88bbb5-bae1-4930-acb2-acac3386a366/node-exporter/0.log" Jul 13 22:46:36.811047 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:36.811023 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7s9sr_5c88bbb5-bae1-4930-acb2-acac3386a366/kube-rbac-proxy/0.log" Jul 13 22:46:36.838981 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:36.838919 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7s9sr_5c88bbb5-bae1-4930-acb2-acac3386a366/init-textfile/0.log" Jul 13 22:46:36.932972 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:36.932936 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mbdmm_e493819d-9f49-421e-a534-9579285072be/kube-rbac-proxy-main/0.log" Jul 13 22:46:36.954740 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:36.954715 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mbdmm_e493819d-9f49-421e-a534-9579285072be/kube-rbac-proxy-self/0.log" Jul 13 22:46:36.976082 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:36.976060 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mbdmm_e493819d-9f49-421e-a534-9579285072be/openshift-state-metrics/0.log" Jul 13 22:46:37.281134 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:37.281112 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c77fb7c4-k7wcb_3eab8f0c-317c-4a8d-9682-3084d30974d9/telemeter-client/0.log" Jul 13 22:46:37.305468 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:37.305446 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c77fb7c4-k7wcb_3eab8f0c-317c-4a8d-9682-3084d30974d9/reload/0.log" Jul 13 22:46:37.326893 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:37.326872 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c77fb7c4-k7wcb_3eab8f0c-317c-4a8d-9682-3084d30974d9/kube-rbac-proxy/0.log" Jul 13 22:46:38.258778 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.258751 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk"] Jul 13 22:46:38.259170 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.259116 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b15880f-ada5-4c3c-bfd2-ddab40473d9d" containerName="maas-api" Jul 13 22:46:38.259170 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.259129 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b15880f-ada5-4c3c-bfd2-ddab40473d9d" containerName="maas-api" Jul 13 22:46:38.259170 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.259139 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8472184c-0683-4c68-a8b7-65c81ad2043a" containerName="revoke-keys" Jul 13 22:46:38.259170 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.259146 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8472184c-0683-4c68-a8b7-65c81ad2043a" containerName="revoke-keys" Jul 13 22:46:38.259299 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.259201 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b15880f-ada5-4c3c-bfd2-ddab40473d9d" containerName="maas-api" Jul 13 22:46:38.259299 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.259210 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8472184c-0683-4c68-a8b7-65c81ad2043a" containerName="revoke-keys" Jul 13 22:46:38.262088 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.262071 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.264408 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.264383 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5tksb\"/\"openshift-service-ca.crt\"" Jul 13 22:46:38.264519 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.264392 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5tksb\"/\"kube-root-ca.crt\"" Jul 13 22:46:38.264957 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.264942 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5tksb\"/\"default-dockercfg-b7v8x\"" Jul 13 22:46:38.280539 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.280517 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk"] Jul 13 22:46:38.360614 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.360584 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-proc\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.360752 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.360646 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-lib-modules\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.360752 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.360682 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-podres\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.360752 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.360722 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgtkk\" (UniqueName: \"kubernetes.io/projected/77dc9ca7-3207-4df1-b008-f21c10f67c29-kube-api-access-pgtkk\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.360923 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.360753 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-sys\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.461942 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.461909 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgtkk\" (UniqueName: \"kubernetes.io/projected/77dc9ca7-3207-4df1-b008-f21c10f67c29-kube-api-access-pgtkk\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.462093 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.461952 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-sys\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.462093 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.461995 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-proc\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.462093 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.462050 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-lib-modules\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.462093 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.462085 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-podres\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.462295 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.462104 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-sys\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.462295 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.462114 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-proc\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.462295 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.462201 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-podres\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.462295 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.462224 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77dc9ca7-3207-4df1-b008-f21c10f67c29-lib-modules\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.470582 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.470556 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgtkk\" (UniqueName: \"kubernetes.io/projected/77dc9ca7-3207-4df1-b008-f21c10f67c29-kube-api-access-pgtkk\") pod \"perf-node-gather-daemonset-v9ndk\" (UID: \"77dc9ca7-3207-4df1-b008-f21c10f67c29\") " pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.571951 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.571885 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.592675 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.592654 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-9sg29_bec934ce-60f6-47d0-8e29-c4ac675f1225/networking-console-plugin/0.log" Jul 13 22:46:38.698082 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.698057 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk"] Jul 13 22:46:38.700199 ip-10-0-143-97 kubenswrapper[2570]: W0713 22:46:38.700172 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod77dc9ca7_3207_4df1_b008_f21c10f67c29.slice/crio-03cf0bd95fb3079e68714347106c9fcc571732815604cdf3ec8d5fbbec995161 WatchSource:0}: Error finding container 03cf0bd95fb3079e68714347106c9fcc571732815604cdf3ec8d5fbbec995161: Status 404 returned error can't find the container with id 03cf0bd95fb3079e68714347106c9fcc571732815604cdf3ec8d5fbbec995161 Jul 13 22:46:38.980795 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.980760 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" event={"ID":"77dc9ca7-3207-4df1-b008-f21c10f67c29","Type":"ContainerStarted","Data":"c2adffb651c0775d227453529d1d30605aa5b756c0c8099c6b1b52c3330dcfb2"} Jul 13 22:46:38.981018 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.980812 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" event={"ID":"77dc9ca7-3207-4df1-b008-f21c10f67c29","Type":"ContainerStarted","Data":"03cf0bd95fb3079e68714347106c9fcc571732815604cdf3ec8d5fbbec995161"} Jul 13 22:46:38.981018 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.980911 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:38.997474 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:38.997427 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" podStartSLOduration=0.997411654 podStartE2EDuration="997.411654ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 22:46:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:46:38.99509262 +0000 UTC m=+2808.177936851" watchObservedRunningTime="2026-07-13 22:46:38.997411654 +0000 UTC m=+2808.180255885" Jul 13 22:46:39.682061 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:39.682030 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69675fd648-7vfh9_adc89823-d927-4d71-9136-c31d17bed90d/console/0.log" Jul 13 22:46:39.714114 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:39.714092 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-2cfgx_0b9017b7-7904-4479-bde7-bf0075a083c4/download-server/0.log" Jul 13 22:46:41.292866 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:41.292822 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qsg7t_0ca89863-2ae9-49cb-8909-abbc0a5b6bb1/dns/0.log" Jul 13 22:46:41.312296 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:41.312270 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qsg7t_0ca89863-2ae9-49cb-8909-abbc0a5b6bb1/kube-rbac-proxy/0.log" Jul 13 22:46:41.392496 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:41.392471 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c9t96_8c06f173-22eb-4265-92c2-45838d12f578/dns-node-resolver/0.log" Jul 13 22:46:41.953580 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:41.953550 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5d77b9978f-h64zc_4e940fa0-f70d-472a-947b-e85e4ecd07c6/registry/0.log" Jul 13 22:46:42.014145 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:42.014118 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nvk85_4fd817a9-f4c3-443d-a8f7-02820f6c426f/node-ca/0.log" Jul 13 22:46:43.015665 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:43.015636 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-d6ljs_3071b09b-3802-4a16-b299-2e04458b8ecd/discovery/0.log" Jul 13 22:46:43.172865 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:43.172816 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-8q5b7_05c0b422-638c-4b6f-a42e-c75fe345e9d7/payload-pre-processing/0.log" Jul 13 22:46:43.201533 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:43.201504 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-5czcx_a0e46651-ae37-4f23-862d-a628630d6e40/payload-processing/0.log" Jul 13 22:46:43.900106 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:43.900071 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6b8db_bf916935-4397-4211-a340-bdaa676732eb/serve-healthcheck-canary/0.log" Jul 13 22:46:44.527019 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:44.526990 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-srjs9_99a7c1d6-e6d3-4869-a624-6eabb61e9da3/kube-rbac-proxy/0.log" Jul 13 22:46:44.568671 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:44.568647 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-srjs9_99a7c1d6-e6d3-4869-a624-6eabb61e9da3/exporter/0.log" Jul 13 22:46:44.599738 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:44.599716 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-srjs9_99a7c1d6-e6d3-4869-a624-6eabb61e9da3/extractor/0.log" Jul 13 22:46:44.995845 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:44.995819 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5tksb/perf-node-gather-daemonset-v9ndk" Jul 13 22:46:47.787560 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:47.787524 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6dc46d8554-mq6tm_3944875c-dd62-4f5e-9e75-c3cf8dbe69bc/manager/0.log" Jul 13 22:46:48.073734 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:48.073661 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67ddb65fc-vn66g_05edc547-1871-4cdc-8770-d18c09e2dba2/manager/0.log" Jul 13 22:46:48.092700 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:48.092672 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67ddb65fc-vn66g_05edc547-1871-4cdc-8770-d18c09e2dba2/copy-manifests/0.log" Jul 13 22:46:48.998954 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:48.998900 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-d6ljs_3071b09b-3802-4a16-b299-2e04458b8ecd/discovery/0.log" Jul 13 22:46:49.126103 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:49.126069 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-8q5b7_05c0b422-638c-4b6f-a42e-c75fe345e9d7/payload-pre-processing/0.log" Jul 13 22:46:49.146604 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:49.146553 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-5czcx_a0e46651-ae37-4f23-862d-a628630d6e40/payload-processing/0.log" Jul 13 22:46:49.403117 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:49.403089 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-88cc59766-7b2r8_2648c485-2db2-4d6b-9e35-8f9be182b7d6/manager/0.log" Jul 13 22:46:50.499496 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:50.499469 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-d6ljs_3071b09b-3802-4a16-b299-2e04458b8ecd/discovery/0.log" Jul 13 22:46:50.639202 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:50.639162 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-8q5b7_05c0b422-638c-4b6f-a42e-c75fe345e9d7/payload-pre-processing/0.log" Jul 13 22:46:50.664992 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:50.664962 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-5czcx_a0e46651-ae37-4f23-862d-a628630d6e40/payload-processing/0.log" Jul 13 22:46:54.273514 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:54.273484 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-vh2hl_b29b53f5-6c35-49a8-9c33-9619b64a9811/migrator/0.log" Jul 13 22:46:54.292987 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:54.292966 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-vh2hl_b29b53f5-6c35-49a8-9c33-9619b64a9811/graceful-termination/0.log" Jul 13 22:46:55.754088 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:55.754063 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p4sz_b9418498-5970-4196-a734-4f64df7d4168/kube-multus-additional-cni-plugins/0.log" Jul 13 22:46:55.775602 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:55.775571 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p4sz_b9418498-5970-4196-a734-4f64df7d4168/egress-router-binary-copy/0.log" Jul 13 22:46:55.794734 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:55.794715 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p4sz_b9418498-5970-4196-a734-4f64df7d4168/cni-plugins/0.log" Jul 13 22:46:55.817049 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:55.817022 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p4sz_b9418498-5970-4196-a734-4f64df7d4168/bond-cni-plugin/0.log" Jul 13 22:46:55.840258 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:55.840242 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p4sz_b9418498-5970-4196-a734-4f64df7d4168/routeoverride-cni/0.log" Jul 13 22:46:55.860048 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:55.860032 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p4sz_b9418498-5970-4196-a734-4f64df7d4168/whereabouts-cni-bincopy/0.log" Jul 13 22:46:55.880492 ip-10-0-143-97 kubenswrapper[2570]: I0713 22:46:55.880470 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p4sz_b9418498-5970-4196-a734-4f64df7d4168/whereabouts-cni/0.log"