Jul 17 21:16:59.117810 ip-10-0-131-188 systemd[1]: Starting Kubernetes Kubelet... Jul 17 21:16:59.480077 ip-10-0-131-188 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 21:16:59.480077 ip-10-0-131-188 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 17 21:16:59.480077 ip-10-0-131-188 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 21:16:59.480077 ip-10-0-131-188 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 17 21:16:59.480077 ip-10-0-131-188 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 21:16:59.480793 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.480140 2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 17 21:16:59.484578 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484561 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 21:16:59.484578 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484578 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484582 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484585 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484590 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484593 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484596 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484598 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484601 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484603 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484606 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484609 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484611 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484614 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484616 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484619 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484621 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484624 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484626 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484629 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484631 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 21:16:59.484645 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484633 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484636 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484638 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484641 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484643 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484646 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484649 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484655 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484658 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484661 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484663 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484665 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484668 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484670 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484672 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484675 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484677 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484680 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484682 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484685 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 21:16:59.485118 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484689 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484692 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484695 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484697 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484700 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484702 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484704 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484708 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484710 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484712 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484715 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484717 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484721 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484724 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484726 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484729 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484731 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484734 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484736 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484739 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 21:16:59.485613 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484742 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484744 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484747 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484749 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484752 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484754 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484757 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484759 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484762 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484764 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484766 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484769 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484771 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484774 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484776 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484778 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484781 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484784 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484786 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 21:16:59.486091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484791 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484795 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484799 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484802 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484805 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484808 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484812 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484815 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484818 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484821 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484824 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484826 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484829 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484832 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484834 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484837 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484840 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484842 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.484845 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 21:16:59.486564 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485528 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485534 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485537 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485539 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485558 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485561 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485564 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485566 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485569 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485571 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485574 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485576 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485579 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485581 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485585 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485588 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485591 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485593 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485597 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 21:16:59.487021 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485602 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485605 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485608 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485610 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485613 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485615 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485618 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485621 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485623 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485626 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485628 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485631 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485634 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485636 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485639 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485641 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485644 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485646 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485649 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485651 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 21:16:59.487499 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485653 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485656 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485658 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485661 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485665 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485668 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485671 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485674 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485677 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485682 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485685 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485688 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485692 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485695 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485697 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485700 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485702 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485705 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485707 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 21:16:59.488005 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485709 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485712 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485714 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485717 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485719 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485722 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485724 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485726 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485729 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485731 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485734 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485736 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485738 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485741 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485743 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485745 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485748 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485750 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485752 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 21:16:59.488482 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485755 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485757 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485760 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485763 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485765 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485768 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485772 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485775 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485778 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485780 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485782 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485785 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485787 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485790 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485793 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485795 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485797 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485800 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485802 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485848 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 21:16:59.488965 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485852 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.485855 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486218 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486223 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486226 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486228 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486231 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486233 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486236 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486238 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486241 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486243 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486246 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486248 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486251 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486254 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486257 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486259 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486262 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 21:16:59.489477 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486264 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486267 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486269 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486272 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486274 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486276 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486279 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486282 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486284 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486286 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486290 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486294 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486297 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486299 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486302 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486304 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486306 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486309 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486311 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486314 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 21:16:59.489951 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486316 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486318 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486321 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486323 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486326 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486328 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486330 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486333 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486336 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486339 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486341 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486347 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486350 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486352 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486355 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486357 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486359 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486362 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486364 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486367 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 21:16:59.490422 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486369 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486372 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486374 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486376 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486379 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486381 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486384 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486386 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486389 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486391 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486393 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486396 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486398 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486401 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486403 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486405 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486409 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486411 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486414 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486416 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 21:16:59.490967 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486419 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486422 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486425 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486430 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486434 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486438 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486441 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486444 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486447 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486449 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486453 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486456 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486459 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486461 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486464 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486466 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486469 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486471 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486473 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 21:16:59.491442 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486476 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486478 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.486481 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486568 2554 flags.go:64] FLAG: --address="0.0.0.0" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486576 2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486584 2554 flags.go:64] FLAG: --anonymous-auth="true" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486588 2554 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486593 2554 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486598 2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486606 2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486612 2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486618 2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486621 2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486629 2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486634 2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486637 2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486642 2554 flags.go:64] FLAG: --cgroup-root="" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486645 2554 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486647 2554 flags.go:64] FLAG: --client-ca-file="" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486650 2554 flags.go:64] FLAG: --cloud-provider="external" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486653 2554 flags.go:64] FLAG: --cluster-dns="[]" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486657 2554 flags.go:64] FLAG: --cluster-domain="" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486660 2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486663 2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 17 21:16:59.491904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486666 2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486669 2554 flags.go:64] FLAG: --container-log-max-files="5" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486673 2554 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486676 2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486678 2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486681 2554 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486685 2554 flags.go:64] FLAG: --contention-profiling="false" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486687 2554 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486690 2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486693 2554 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486696 2554 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486700 2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486702 2554 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486705 2554 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486708 2554 flags.go:64] FLAG: --enable-load-reader="false" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486710 2554 flags.go:64] FLAG: --enable-server="true" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486713 2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486717 2554 flags.go:64] FLAG: --event-burst="100" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486720 2554 flags.go:64] FLAG: --event-qps="50" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486722 2554 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486725 2554 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486735 2554 flags.go:64] FLAG: --eviction-hard="" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486738 2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486741 2554 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486744 2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 17 21:16:59.492509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486747 2554 flags.go:64] FLAG: --eviction-soft="" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486750 2554 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486753 2554 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486756 2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486758 2554 flags.go:64] FLAG: --experimental-mounter-path="" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486761 2554 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486764 2554 flags.go:64] FLAG: --fail-swap-on="true" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486767 2554 flags.go:64] FLAG: --feature-gates="" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486770 2554 flags.go:64] FLAG: --file-check-frequency="20s" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486773 2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486776 2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486779 2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486781 2554 flags.go:64] FLAG: --healthz-port="10248" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486784 2554 flags.go:64] FLAG: --help="false" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486787 2554 flags.go:64] FLAG: --hostname-override="ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486790 2554 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486793 2554 flags.go:64] FLAG: --http-check-frequency="20s" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486795 2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486798 2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486802 2554 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486805 2554 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486807 2554 flags.go:64] FLAG: --image-service-endpoint="" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486810 2554 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 17 21:16:59.493122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486813 2554 flags.go:64] FLAG: --kube-api-burst="100" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486815 2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486818 2554 flags.go:64] FLAG: --kube-api-qps="50" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486820 2554 flags.go:64] FLAG: --kube-reserved="" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486823 2554 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486827 2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486830 2554 flags.go:64] FLAG: --kubelet-cgroups="" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486832 2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486835 2554 flags.go:64] FLAG: --lock-file="" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486838 2554 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486844 2554 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486847 2554 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486852 2554 flags.go:64] FLAG: --log-json-split-stream="false" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486854 2554 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486857 2554 flags.go:64] FLAG: --log-text-split-stream="false" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486860 2554 flags.go:64] FLAG: --logging-format="text" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486862 2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486866 2554 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486868 2554 flags.go:64] FLAG: --manifest-url="" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486871 2554 flags.go:64] FLAG: --manifest-url-header="" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486875 2554 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486878 2554 flags.go:64] FLAG: --max-open-files="1000000" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486882 2554 flags.go:64] FLAG: --max-pods="110" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486884 2554 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486887 2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 17 21:16:59.493738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486890 2554 flags.go:64] FLAG: --memory-manager-policy="None" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486892 2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486895 2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486898 2554 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486901 2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486907 2554 flags.go:64] FLAG: --node-status-max-images="50" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486910 2554 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486913 2554 flags.go:64] FLAG: --oom-score-adj="-999" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486915 2554 flags.go:64] FLAG: --pod-cidr="" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486918 2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486923 2554 flags.go:64] FLAG: --pod-manifest-path="" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486926 2554 flags.go:64] FLAG: --pod-max-pids="-1" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486929 2554 flags.go:64] FLAG: --pods-per-core="0" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486932 2554 flags.go:64] FLAG: --port="10250" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486935 2554 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486938 2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0aee31f94d1718fcc" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486941 2554 flags.go:64] FLAG: --qos-reserved="" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486944 2554 flags.go:64] FLAG: --read-only-port="10255" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486947 2554 flags.go:64] FLAG: --register-node="true" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486950 2554 flags.go:64] FLAG: --register-with-taints="" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486953 2554 flags.go:64] FLAG: --registry-burst="10" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486956 2554 flags.go:64] FLAG: --registry-qps="5" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486958 2554 flags.go:64] FLAG: --reserved-cpus="" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486961 2554 flags.go:64] FLAG: --reserved-memory="" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486964 2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 17 21:16:59.494351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486967 2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486969 2554 flags.go:64] FLAG: --rotate-certificates="false" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486972 2554 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486975 2554 flags.go:64] FLAG: --runonce="false" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486977 2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486980 2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486983 2554 flags.go:64] FLAG: --seccomp-default="false" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486986 2554 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486988 2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486991 2554 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486994 2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486997 2554 flags.go:64] FLAG: --storage-driver-password="root" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.486999 2554 flags.go:64] FLAG: --storage-driver-secure="false" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487002 2554 flags.go:64] FLAG: --storage-driver-table="stats" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487005 2554 flags.go:64] FLAG: --storage-driver-user="root" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487007 2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487010 2554 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487013 2554 flags.go:64] FLAG: --system-cgroups="" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487016 2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487021 2554 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487025 2554 flags.go:64] FLAG: --tls-cert-file="" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487030 2554 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487034 2554 flags.go:64] FLAG: --tls-min-version="" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487037 2554 flags.go:64] FLAG: --tls-private-key-file="" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487040 2554 flags.go:64] FLAG: --topology-manager-policy="none" Jul 17 21:16:59.495010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487042 2554 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487046 2554 flags.go:64] FLAG: --topology-manager-scope="container" Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487049 2554 flags.go:64] FLAG: --v="2" Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487052 2554 flags.go:64] FLAG: --version="false" Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487056 2554 flags.go:64] FLAG: --vmodule="" Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487060 2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.487063 2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487178 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487182 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487185 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487187 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487192 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487195 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487198 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487201 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487204 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487206 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487209 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487211 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487214 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487216 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 21:16:59.495621 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487218 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487221 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487223 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487226 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487228 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487231 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487234 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487238 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487240 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487243 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487246 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487249 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487252 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487256 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487260 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487263 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487265 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487268 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487271 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 21:16:59.496171 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487273 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487275 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487278 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487280 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487283 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487285 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487288 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487290 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487293 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487295 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487297 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487300 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487302 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487304 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487307 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487309 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487312 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487314 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487316 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487320 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 21:16:59.496640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487323 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487326 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487328 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487331 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487333 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487336 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487338 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487342 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487344 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487347 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487349 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487351 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487354 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487356 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487359 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487362 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487364 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487367 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487369 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487371 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 21:16:59.497121 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487374 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487376 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487379 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487381 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487383 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487386 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487388 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487390 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487393 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487395 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487397 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487401 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487404 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487407 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487409 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487412 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487414 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487416 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487419 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 21:16:59.497679 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487422 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487425 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487427 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487430 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487432 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487435 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.487437 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.488259 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.494470 2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.494484 2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.494502 2554 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.494507 2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494598 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494604 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494607 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 21:16:59.498122 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494610 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494613 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494616 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494618 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494621 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494624 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494626 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494629 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494633 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494637 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494640 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494644 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494646 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494649 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494651 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494653 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494656 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494658 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494664 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494667 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 21:16:59.498492 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494669 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494672 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494674 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494676 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494680 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494682 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494685 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494688 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494690 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494693 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494696 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494698 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494701 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494703 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494706 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494708 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494711 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494713 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494716 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494718 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 21:16:59.499091 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494721 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494723 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494725 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494728 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494731 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494733 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494736 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494739 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494741 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494743 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494746 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494749 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494751 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494754 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494756 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494758 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494761 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494763 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494766 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494769 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 21:16:59.499637 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494771 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494774 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494776 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494778 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494781 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494783 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494785 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494788 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494790 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494793 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494796 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494799 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494802 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494805 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494808 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494811 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494813 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494816 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494818 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494821 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 21:16:59.500182 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494824 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494826 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494828 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494831 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494833 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494836 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494838 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494841 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494843 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494846 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494848 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494852 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494855 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494858 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494861 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 21:16:59.500701 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494864 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.494869 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494994 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.494999 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495002 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495005 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495008 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495012 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495015 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495018 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495021 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495024 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495027 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495030 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 21:16:59.501173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495033 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495035 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495038 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495040 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495043 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495045 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495047 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495049 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495055 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495058 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495060 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495063 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495065 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495068 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495070 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495073 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495075 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495078 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495080 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 21:16:59.501538 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495091 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495094 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495096 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495099 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495102 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495104 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495106 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495109 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495111 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495114 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495116 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495119 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495121 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495124 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495126 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495128 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495131 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495133 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495135 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495137 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 21:16:59.502073 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495140 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495142 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495148 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495150 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495153 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495155 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495157 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495160 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495162 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495165 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495167 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495169 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495172 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495174 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495177 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495179 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495182 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495184 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495186 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495189 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 21:16:59.502601 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495191 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495193 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495196 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495198 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495200 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495203 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495205 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495207 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495210 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495212 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495214 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495217 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495219 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495221 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495225 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495234 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495237 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495240 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495243 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 21:16:59.503203 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495247 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495250 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495253 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495255 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495258 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495261 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495263 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495266 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:16:59.495269 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.495274 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.495372 2554 server.go:962] "Client rotation is on, will bootstrap in background" Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.499587 2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.500656 2554 server.go:1019] "Starting client certificate rotation" Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.500786 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 21:16:59.504007 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.500835 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 21:16:59.504425 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.502894 2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 21:16:59.508586 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.508559 2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 21:16:59.516513 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.516495 2554 log.go:25] "Validated CRI v1 runtime API" Jul 17 21:16:59.518275 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.518260 2554 log.go:25] "Validated CRI v1 image API" Jul 17 21:16:59.519100 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.519077 2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 17 21:16:59.521196 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.521180 2554 fs.go:135] Filesystem UUIDs: map[158e587c-baa4-4ed4-93e5-66d956d8ebb5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 89e01721-7ede-4e71-a339-b4e04fe38495:/dev/nvme0n1p3] Jul 17 21:16:59.521249 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.521196 2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 17 21:16:59.526679 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.526571 2554 manager.go:216] Machine: {Timestamp:2026-07-17 21:16:59.525222541 +0000 UTC m=+0.294570510 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099486 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b9164e528445d051229e81bef8d8c SystemUUID:ec2b9164-e528-445d-0512-29e81bef8d8c BootID:b226981e-85d4-4d61-ab26-d0bcf7822344 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a8:71:ad:ee:27 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a8:71:ad:ee:27 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:71:81:ae:6f:bc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 17 21:16:59.526679 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.526674 2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 17 21:16:59.526799 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.526745 2554 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 17 21:16:59.529810 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.529785 2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 17 21:16:59.529935 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.529813 2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-188.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 17 21:16:59.530490 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.530481 2554 topology_manager.go:138] "Creating topology manager with none policy" Jul 17 21:16:59.530523 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.530492 2554 container_manager_linux.go:309] "Creating device plugin manager" Jul 17 21:16:59.530523 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.530499 2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 21:16:59.531156 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.531144 2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 21:16:59.532071 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.532058 2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 17 21:16:59.533879 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.533861 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 21:16:59.534399 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.534388 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 17 21:16:59.534513 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.534504 2554 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 17 21:16:59.534570 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.534535 2554 kubelet.go:486] "Attempting to sync node with API server" Jul 17 21:16:59.534570 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.534564 2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 17 21:16:59.534636 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.534579 2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 17 21:16:59.534636 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.534594 2554 kubelet.go:398] "Adding apiserver pod source" Jul 17 21:16:59.534636 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.534604 2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 17 21:16:59.534964 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.534953 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 21:16:59.535009 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.534971 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 21:16:59.537344 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.537326 2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 17 21:16:59.539725 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.539706 2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 17 21:16:59.539796 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.539754 2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 17 21:16:59.540895 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.540877 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 17 21:16:59.540966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.540905 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 17 21:16:59.540966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.540923 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 17 21:16:59.540966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.540932 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 17 21:16:59.540966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.540944 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 17 21:16:59.540966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.540953 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 17 21:16:59.540966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.540965 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 17 21:16:59.541230 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.540977 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 17 21:16:59.541230 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.540988 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 17 21:16:59.541230 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.541040 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 17 21:16:59.541230 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.541057 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 17 21:16:59.541230 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.541068 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 17 21:16:59.541230 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.541096 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 17 21:16:59.541230 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.541106 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 17 21:16:59.544385 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.544370 2554 server.go:1268] "Started kubelet" Jul 17 21:16:59.544531 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.544473 2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 17 21:16:59.544714 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.544671 2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 17 21:16:59.544774 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.544730 2554 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 17 21:16:59.546010 ip-10-0-131-188 systemd[1]: Started Kubernetes Kubelet. Jul 17 21:16:59.547077 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.547059 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 17 21:16:59.547148 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.547090 2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-188.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 17 21:16:59.547148 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.547129 2554 server.go:310] "Adding debug handlers to kubelet server" Jul 17 21:16:59.547245 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.547127 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 17 21:16:59.549365 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.549346 2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 17 21:16:59.550848 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.550830 2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 17 21:16:59.556891 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.554827 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-188.ec2.internal.18c33055207e0a92 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-188.ec2.internal,UID:ip-10-0-131-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-188.ec2.internal,},FirstTimestamp:2026-07-17 21:16:59.544332946 +0000 UTC m=+0.313680916,LastTimestamp:2026-07-17 21:16:59.544332946 +0000 UTC m=+0.313680916,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-188.ec2.internal,}" Jul 17 21:16:59.559475 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.559462 2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 17 21:16:59.559523 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.559480 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 17 21:16:59.559625 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.559608 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:16:59.560042 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.559983 2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 17 21:16:59.560042 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.559985 2554 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 17 21:16:59.560042 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.560029 2554 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 17 21:16:59.560175 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.560137 2554 reconstruct.go:97] "Volume reconstruction finished" Jul 17 21:16:59.560175 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.560147 2554 reconciler.go:29] "Reconciler: start to sync state" Jul 17 21:16:59.560572 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.560533 2554 factory.go:55] Registering systemd factory Jul 17 21:16:59.560657 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.560577 2554 factory.go:223] Registration of the systemd container factory successfully Jul 17 21:16:59.560880 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.560865 2554 factory.go:153] Registering CRI-O factory Jul 17 21:16:59.560957 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.560883 2554 factory.go:223] Registration of the crio container factory successfully Jul 17 21:16:59.560957 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.560939 2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 17 21:16:59.561034 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.560963 2554 factory.go:103] Registering Raw factory Jul 17 21:16:59.561034 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.560979 2554 manager.go:1196] Started watching for new ooms in manager Jul 17 21:16:59.561409 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.561397 2554 manager.go:349] Starting recovery of all containers Jul 17 21:16:59.562164 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.562120 2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 17 21:16:59.569714 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.569684 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 17 21:16:59.572654 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.572631 2554 manager.go:354] Recovery completed Jul 17 21:16:59.574343 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.574311 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 17 21:16:59.577995 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.577978 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 21:16:59.580672 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.580654 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientMemory" Jul 17 21:16:59.580738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.580687 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 21:16:59.580738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.580701 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientPID" Jul 17 21:16:59.581224 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.581210 2554 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 17 21:16:59.581224 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.581217 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 21:16:59.581224 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.581222 2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 17 21:16:59.581372 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.581237 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 17 21:16:59.581999 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.581985 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientMemory" Jul 17 21:16:59.582054 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.582011 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 21:16:59.582054 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.582023 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientPID" Jul 17 21:16:59.582610 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.582599 2554 policy_none.go:49] "None policy: Start" Jul 17 21:16:59.582649 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.582614 2554 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 17 21:16:59.582649 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.582624 2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 17 21:16:59.583532 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.583522 2554 policy_none.go:47] "Start" Jul 17 21:16:59.590752 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.590688 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-188.ec2.internal.18c3305522a886dc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-188.ec2.internal,UID:ip-10-0-131-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-188.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-188.ec2.internal,},FirstTimestamp:2026-07-17 21:16:59.580671708 +0000 UTC m=+0.350019680,LastTimestamp:2026-07-17 21:16:59.580671708 +0000 UTC m=+0.350019680,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-188.ec2.internal,}" Jul 17 21:16:59.592816 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.592757 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-188.ec2.internal.18c3305522a8e196 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-188.ec2.internal,UID:ip-10-0-131-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-188.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-188.ec2.internal,},FirstTimestamp:2026-07-17 21:16:59.580694934 +0000 UTC m=+0.350042906,LastTimestamp:2026-07-17 21:16:59.580694934 +0000 UTC m=+0.350042906,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-188.ec2.internal,}" Jul 17 21:16:59.601484 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.601462 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j85br" Jul 17 21:16:59.605846 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.605783 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-188.ec2.internal.18c3305522a90a0d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-188.ec2.internal,UID:ip-10-0-131-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-188.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-188.ec2.internal,},FirstTimestamp:2026-07-17 21:16:59.580705293 +0000 UTC m=+0.350053265,LastTimestamp:2026-07-17 21:16:59.580705293 +0000 UTC m=+0.350053265,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-188.ec2.internal,}" Jul 17 21:16:59.613236 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.613217 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j85br" Jul 17 21:16:59.618272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.618259 2554 manager.go:337] "Starting Device Plugin manager" Jul 17 21:16:59.618337 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.618289 2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 17 21:16:59.618337 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.618303 2554 server.go:87] "Starting device plugin registration server" Jul 17 21:16:59.618593 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.618580 2554 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 17 21:16:59.618640 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.618595 2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 17 21:16:59.618701 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.618685 2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 17 21:16:59.618773 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.618756 2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 17 21:16:59.618773 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.618768 2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 17 21:16:59.619363 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.619330 2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 17 21:16:59.619426 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.619393 2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:16:59.713316 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.713290 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 17 21:16:59.714330 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.713332 2554 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 17 21:16:59.714330 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.713356 2554 kubelet.go:2448] "Starting kubelet main sync loop" Jul 17 21:16:59.714330 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.713392 2554 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 17 21:16:59.716666 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.716651 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 21:16:59.719034 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.719021 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 21:16:59.719786 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.719772 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientMemory" Jul 17 21:16:59.719864 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.719804 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 21:16:59.719864 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.719814 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientPID" Jul 17 21:16:59.719864 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.719834 2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.729799 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.729778 2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.729877 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.729801 2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-188.ec2.internal\": node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:16:59.773216 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.773193 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:16:59.814323 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.814295 2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-188.ec2.internal"] Jul 17 21:16:59.814450 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.814384 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 21:16:59.815865 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.815850 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientMemory" Jul 17 21:16:59.815950 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.815884 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 21:16:59.815950 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.815898 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientPID" Jul 17 21:16:59.816919 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.816904 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 21:16:59.817019 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.817005 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.817072 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.817033 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 21:16:59.817644 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.817629 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientMemory" Jul 17 21:16:59.817644 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.817638 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientMemory" Jul 17 21:16:59.817752 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.817655 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 21:16:59.817752 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.817657 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 21:16:59.817752 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.817666 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientPID" Jul 17 21:16:59.817752 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.817669 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientPID" Jul 17 21:16:59.818706 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.818691 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.818784 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.818721 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 21:16:59.819435 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.819419 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientMemory" Jul 17 21:16:59.819502 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.819446 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 21:16:59.819502 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.819460 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientPID" Jul 17 21:16:59.850362 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.850341 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-188.ec2.internal\" not found" node="ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.854995 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.854979 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-188.ec2.internal\" not found" node="ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.861748 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.861721 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17a5673d75761b16d9c36066d01f1326-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal\" (UID: \"17a5673d75761b16d9c36066d01f1326\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.861878 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.861758 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fba93f9b97932f873e087f216da95d10-config\") pod \"kube-apiserver-proxy-ip-10-0-131-188.ec2.internal\" (UID: \"fba93f9b97932f873e087f216da95d10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.861878 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.861774 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/17a5673d75761b16d9c36066d01f1326-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal\" (UID: \"17a5673d75761b16d9c36066d01f1326\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.873722 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.873704 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:16:59.962132 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.962099 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17a5673d75761b16d9c36066d01f1326-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal\" (UID: \"17a5673d75761b16d9c36066d01f1326\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.962132 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.962064 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17a5673d75761b16d9c36066d01f1326-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal\" (UID: \"17a5673d75761b16d9c36066d01f1326\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.962345 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.962180 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fba93f9b97932f873e087f216da95d10-config\") pod \"kube-apiserver-proxy-ip-10-0-131-188.ec2.internal\" (UID: \"fba93f9b97932f873e087f216da95d10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.962345 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.962204 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/17a5673d75761b16d9c36066d01f1326-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal\" (UID: \"17a5673d75761b16d9c36066d01f1326\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.962345 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.962235 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/17a5673d75761b16d9c36066d01f1326-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal\" (UID: \"17a5673d75761b16d9c36066d01f1326\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.962345 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:16:59.962248 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fba93f9b97932f873e087f216da95d10-config\") pod \"kube-apiserver-proxy-ip-10-0-131-188.ec2.internal\" (UID: \"fba93f9b97932f873e087f216da95d10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-188.ec2.internal" Jul 17 21:16:59.974188 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:16:59.974159 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:17:00.074955 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:00.074883 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:17:00.154103 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.154080 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" Jul 17 21:17:00.157903 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.157649 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-188.ec2.internal" Jul 17 21:17:00.175272 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:00.175245 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:17:00.275910 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:00.275865 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:17:00.376403 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:00.376330 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:17:00.477152 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:00.477120 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:17:00.500307 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.500279 2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 17 21:17:00.500798 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.500408 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 21:17:00.519597 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.519576 2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 21:17:00.577803 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:00.577776 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:17:00.613961 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.613941 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 21:17:00.615591 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.615555 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientMemory" Jul 17 21:17:00.615591 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.615587 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 21:17:00.615727 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.615599 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeHasSufficientPID" Jul 17 21:17:00.616788 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.616759 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 21:11:59 +0000 UTC" deadline="2028-04-17 22:09:36.47780046 +0000 UTC" Jul 17 21:17:00.616864 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.616788 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15360h52m35.861015256s" Jul 17 21:17:00.678792 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:00.678730 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:17:00.717755 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:00.717710 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfba93f9b97932f873e087f216da95d10.slice/crio-f2f01711578986b3b916a0c4d4afd5f8cfadd32790e718fee62c198fc297827c WatchSource:0}: Error finding container f2f01711578986b3b916a0c4d4afd5f8cfadd32790e718fee62c198fc297827c: Status 404 returned error can't find the container with id f2f01711578986b3b916a0c4d4afd5f8cfadd32790e718fee62c198fc297827c Jul 17 21:17:00.718013 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:00.717993 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17a5673d75761b16d9c36066d01f1326.slice/crio-b22843456d3ccaf3890829db505150d43dbf7b69ab7289e652c7a5393db494a3 WatchSource:0}: Error finding container b22843456d3ccaf3890829db505150d43dbf7b69ab7289e652c7a5393db494a3: Status 404 returned error can't find the container with id b22843456d3ccaf3890829db505150d43dbf7b69ab7289e652c7a5393db494a3 Jul 17 21:17:00.722182 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.722167 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 21:17:00.779816 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:00.779778 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-188.ec2.internal\" not found" Jul 17 21:17:00.830846 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.830819 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 21:17:00.860216 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.860192 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" Jul 17 21:17:00.882200 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.882171 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 21:17:00.883298 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.883284 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-188.ec2.internal" Jul 17 21:17:00.897336 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:00.897312 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 21:17:01.055873 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.055843 2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 21:17:01.535232 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.535199 2554 apiserver.go:52] "Watching apiserver" Jul 17 21:17:01.545022 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.544992 2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 17 21:17:01.547703 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.547670 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-rjxhs","openshift-multus/network-metrics-daemon-6w8mk","openshift-network-diagnostics/network-check-target-7cbth","openshift-ovn-kubernetes/ovnkube-node-vf7xx","openshift-cluster-node-tuning-operator/tuned-mndvj","openshift-image-registry/node-ca-6zztr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal","openshift-multus/multus-57924","openshift-multus/multus-additional-cni-plugins-fkkg4","openshift-network-operator/iptables-alerter-z84hz","kube-system/global-pull-secret-syncer-79tgn","kube-system/konnectivity-agent-grpkq","kube-system/kube-apiserver-proxy-ip-10-0-131-188.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm"] Jul 17 21:17:01.549291 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.549270 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z84hz" Jul 17 21:17:01.551131 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.551077 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:01.551234 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:01.551190 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:01.552613 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.552576 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:01.552711 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:01.552661 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:01.552771 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.552735 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 17 21:17:01.552857 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.552841 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z7bfd\"" Jul 17 21:17:01.553093 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.553073 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 17 21:17:01.553240 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.553079 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 17 21:17:01.553689 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.553668 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.555455 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.555438 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.556246 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.556227 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 17 21:17:01.556573 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.556542 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 17 21:17:01.556573 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.556570 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tk7km\"" Jul 17 21:17:01.556733 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.556609 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6zztr" Jul 17 21:17:01.556942 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.556925 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 17 21:17:01.557896 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.557876 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.559357 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.558369 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 17 21:17:01.559357 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.558490 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 17 21:17:01.559357 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.558630 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 17 21:17:01.559357 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.558716 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-56bcr\"" Jul 17 21:17:01.559357 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.558845 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 17 21:17:01.559357 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.558947 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 17 21:17:01.559868 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.559365 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.560616 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.560576 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 17 21:17:01.560913 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.560893 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:01.560998 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:01.560969 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:01.561987 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.561896 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rjxhs" Jul 17 21:17:01.563305 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.563287 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-57924" Jul 17 21:17:01.563508 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.563489 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 17 21:17:01.563932 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.563918 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 17 21:17:01.564231 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.564216 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 17 21:17:01.564701 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.564679 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:01.565736 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.565715 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kgl8q\"" Jul 17 21:17:01.565832 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.565811 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zxg6r\"" Jul 17 21:17:01.565960 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.565944 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 17 21:17:01.566129 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.566092 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 17 21:17:01.570613 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.567666 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 17 21:17:01.570613 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.568060 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-574l5\"" Jul 17 21:17:01.570613 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.568303 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 17 21:17:01.570613 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.568524 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 17 21:17:01.570613 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.568785 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 17 21:17:01.570613 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.569607 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 17 21:17:01.570613 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.569825 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 17 21:17:01.570613 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.570289 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f9rlz\"" Jul 17 21:17:01.571443 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.571139 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.571849 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.572555 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-modprobe-d\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.572609 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-registration-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.572645 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-device-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.572675 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-cni-bin\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.572706 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-lib-modules\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.572850 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/82b932c6-788d-4a01-bdcb-c8c9b7303984-serviceca\") pod \"node-ca-6zztr\" (UID: \"82b932c6-788d-4a01-bdcb-c8c9b7303984\") " pod="openshift-image-registry/node-ca-6zztr" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.572876 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-run-systemd\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.572908 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-run-ovn\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.572942 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-node-log\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.572973 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-sys\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573024 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573054 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573141 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-var-lib-kubelet\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573166 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-tuned\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573206 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-system-cni-dir\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573274 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b870da3-94e0-4301-8be9-17174f8a012a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.574963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573310 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573340 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-run-netns\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573428 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f9e4986-4f46-4007-b625-b49c6e766dec-ovnkube-config\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573590 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xntmp\"" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573823 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573587 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82b932c6-788d-4a01-bdcb-c8c9b7303984-host\") pod \"node-ca-6zztr\" (UID: \"82b932c6-788d-4a01-bdcb-c8c9b7303984\") " pod="openshift-image-registry/node-ca-6zztr" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.573881 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574222 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-log-socket\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574295 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-run-ovn-kubernetes\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574382 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f9e4986-4f46-4007-b625-b49c6e766dec-ovnkube-script-lib\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574409 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l49n7\" (UniqueName: \"kubernetes.io/projected/17977676-3dc1-41d5-8645-2a040631675b-kube-api-access-l49n7\") pod \"iptables-alerter-z84hz\" (UID: \"17977676-3dc1-41d5-8645-2a040631675b\") " pod="openshift-network-operator/iptables-alerter-z84hz" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574483 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx4bb\" (UniqueName: \"kubernetes.io/projected/82b932c6-788d-4a01-bdcb-c8c9b7303984-kube-api-access-gx4bb\") pod \"node-ca-6zztr\" (UID: \"82b932c6-788d-4a01-bdcb-c8c9b7303984\") " pod="openshift-image-registry/node-ca-6zztr" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574525 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574635 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-sys-fs\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574666 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f9e4986-4f46-4007-b625-b49c6e766dec-env-overrides\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574716 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s8jp\" (UniqueName: \"kubernetes.io/projected/6b870da3-94e0-4301-8be9-17174f8a012a-kube-api-access-8s8jp\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574773 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwdnl\" (UniqueName: \"kubernetes.io/projected/7507ea3a-57dc-4701-aaf0-5e241ab82b74-kube-api-access-qwdnl\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:01.575769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574818 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17977676-3dc1-41d5-8645-2a040631675b-iptables-alerter-script\") pod \"iptables-alerter-z84hz\" (UID: \"17977676-3dc1-41d5-8645-2a040631675b\") " pod="openshift-network-operator/iptables-alerter-z84hz" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574885 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-sysconfig\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574926 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.574968 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f9e4986-4f46-4007-b625-b49c6e766dec-ovn-node-metrics-cert\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575012 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-cni-netd\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575087 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-host\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575135 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6964db66-5d84-43e3-8f76-e7e23aeb63d3-kubelet-config\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575217 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-sysctl-d\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575272 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4412bdbc-bb2e-485e-bcf2-58166173a080-tmp\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575407 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-socket-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575463 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-slash\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575506 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575557 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt5bp\" (UniqueName: \"kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp\") pod \"network-check-target-7cbth\" (UID: \"98ba8f25-7586-4345-8e2e-4c87629466ca\") " pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575609 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17977676-3dc1-41d5-8645-2a040631675b-host-slash\") pod \"iptables-alerter-z84hz\" (UID: \"17977676-3dc1-41d5-8645-2a040631675b\") " pod="openshift-network-operator/iptables-alerter-z84hz" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575666 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwbd4\" (UniqueName: \"kubernetes.io/projected/44c51362-b659-4fe7-9aed-1d649b89d193-kube-api-access-zwbd4\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575714 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-kubernetes\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.576420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575782 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psn9h\" (UniqueName: \"kubernetes.io/projected/4412bdbc-bb2e-485e-bcf2-58166173a080-kube-api-access-psn9h\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575826 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-os-release\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575865 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s2ww\" (UniqueName: \"kubernetes.io/projected/6f9e4986-4f46-4007-b625-b49c6e766dec-kube-api-access-2s2ww\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575896 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-sysctl-conf\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575921 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-systemd\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.575975 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-cnibin\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.576031 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-etc-openvswitch\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.576064 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b870da3-94e0-4301-8be9-17174f8a012a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.576091 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b870da3-94e0-4301-8be9-17174f8a012a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.576144 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6964db66-5d84-43e3-8f76-e7e23aeb63d3-dbus\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.576173 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-kubelet\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.576221 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-systemd-units\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.576247 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-var-lib-openvswitch\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.576307 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-run-openvswitch\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.576336 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-run\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.577055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.576390 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:01.578793 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.578601 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pdsk9\"" Jul 17 21:17:01.582845 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.582822 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 21:17:01.606272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.606245 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gf6qg" Jul 17 21:17:01.616528 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.616494 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gf6qg" Jul 17 21:17:01.660816 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.660793 2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 17 21:17:01.676606 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676574 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-cnibin\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.676772 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676624 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-os-release\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.676772 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676653 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4eb2215-40df-4faf-9040-0b4a38ed8198-agent-certs\") pod \"konnectivity-agent-grpkq\" (UID: \"f4eb2215-40df-4faf-9040-0b4a38ed8198\") " pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:01.676772 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676679 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-etc-openvswitch\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.676772 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676739 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-cnibin\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.676772 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676741 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b870da3-94e0-4301-8be9-17174f8a012a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.677048 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676807 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b870da3-94e0-4301-8be9-17174f8a012a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.677048 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676834 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6964db66-5d84-43e3-8f76-e7e23aeb63d3-dbus\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:01.677048 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676858 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-kubelet\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.677048 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676890 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-systemd-units\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.677048 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676914 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-var-lib-openvswitch\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.677048 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676941 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-run-openvswitch\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.677048 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676967 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-run\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.677048 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.676990 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:01.677048 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677016 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-modprobe-d\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.677048 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677040 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-registration-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677067 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-device-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677089 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-cni-bin\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677132 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-lib-modules\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677220 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/82b932c6-788d-4a01-bdcb-c8c9b7303984-serviceca\") pod \"node-ca-6zztr\" (UID: \"82b932c6-788d-4a01-bdcb-c8c9b7303984\") " pod="openshift-image-registry/node-ca-6zztr" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677252 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8sz9\" (UniqueName: \"kubernetes.io/projected/768859c6-079b-4947-ad85-d6bf78ac1023-kube-api-access-z8sz9\") pod \"node-resolver-rjxhs\" (UID: \"768859c6-079b-4947-ad85-d6bf78ac1023\") " pod="openshift-dns/node-resolver-rjxhs" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677293 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-run-k8s-cni-cncf-io\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677321 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-run-systemd\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677355 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-run-ovn\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677380 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-node-log\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677385 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b870da3-94e0-4301-8be9-17174f8a012a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677408 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-sys\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677491 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-var-lib-kubelet\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677499 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-etc-openvswitch\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.677506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677508 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-modprobe-d\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677516 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-tuned\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677545 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-system-cni-dir\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677603 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-registration-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677670 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-systemd-units\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677683 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-var-lib-openvswitch\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677709 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b870da3-94e0-4301-8be9-17174f8a012a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677730 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-system-cni-dir\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677715 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-kubelet\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677748 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677785 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-sys\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677783 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-run-netns\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677781 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-cni-bin\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677832 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f9e4986-4f46-4007-b625-b49c6e766dec-ovnkube-config\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677841 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-run-netns\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677851 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677857 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82b932c6-788d-4a01-bdcb-c8c9b7303984-host\") pod \"node-ca-6zztr\" (UID: \"82b932c6-788d-4a01-bdcb-c8c9b7303984\") " pod="openshift-image-registry/node-ca-6zztr" Jul 17 21:17:01.678198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677881 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677912 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-run-netns\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677936 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82b932c6-788d-4a01-bdcb-c8c9b7303984-host\") pod \"node-ca-6zztr\" (UID: \"82b932c6-788d-4a01-bdcb-c8c9b7303984\") " pod="openshift-image-registry/node-ca-6zztr" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677939 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-var-lib-cni-multus\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677873 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-run\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.677978 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8qzw\" (UniqueName: \"kubernetes.io/projected/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-kube-api-access-x8qzw\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678008 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-device-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:01.678037 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678048 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-node-log\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678017 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-run-openvswitch\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678091 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-log-socket\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678152 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-lib-modules\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678199 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-log-socket\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:01.678188 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret podName:6964db66-5d84-43e3-8f76-e7e23aeb63d3 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:02.178146744 +0000 UTC m=+2.947494705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret") pod "global-pull-secret-syncer-79tgn" (UID: "6964db66-5d84-43e3-8f76-e7e23aeb63d3") : object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678230 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-run-ovn-kubernetes\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678261 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f9e4986-4f46-4007-b625-b49c6e766dec-ovnkube-script-lib\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678276 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b870da3-94e0-4301-8be9-17174f8a012a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.679090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678288 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l49n7\" (UniqueName: \"kubernetes.io/projected/17977676-3dc1-41d5-8645-2a040631675b-kube-api-access-l49n7\") pod \"iptables-alerter-z84hz\" (UID: \"17977676-3dc1-41d5-8645-2a040631675b\") " pod="openshift-network-operator/iptables-alerter-z84hz" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678318 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-cnibin\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678348 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx4bb\" (UniqueName: \"kubernetes.io/projected/82b932c6-788d-4a01-bdcb-c8c9b7303984-kube-api-access-gx4bb\") pod \"node-ca-6zztr\" (UID: \"82b932c6-788d-4a01-bdcb-c8c9b7303984\") " pod="openshift-image-registry/node-ca-6zztr" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678344 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-run-ovn-kubernetes\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678387 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678418 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-sys-fs\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678454 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f9e4986-4f46-4007-b625-b49c6e766dec-env-overrides\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678463 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f9e4986-4f46-4007-b625-b49c6e766dec-ovnkube-config\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678485 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s8jp\" (UniqueName: \"kubernetes.io/projected/6b870da3-94e0-4301-8be9-17174f8a012a-kube-api-access-8s8jp\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678522 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwdnl\" (UniqueName: \"kubernetes.io/projected/7507ea3a-57dc-4701-aaf0-5e241ab82b74-kube-api-access-qwdnl\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678551 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17977676-3dc1-41d5-8645-2a040631675b-iptables-alerter-script\") pod \"iptables-alerter-z84hz\" (UID: \"17977676-3dc1-41d5-8645-2a040631675b\") " pod="openshift-network-operator/iptables-alerter-z84hz" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678561 2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678595 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/82b932c6-788d-4a01-bdcb-c8c9b7303984-serviceca\") pod \"node-ca-6zztr\" (UID: \"82b932c6-788d-4a01-bdcb-c8c9b7303984\") " pod="openshift-image-registry/node-ca-6zztr" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:01.678685 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:01.678759 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs podName:7507ea3a-57dc-4701-aaf0-5e241ab82b74 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:02.178724927 +0000 UTC m=+2.948072885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs") pod "network-metrics-daemon-6w8mk" (UID: "7507ea3a-57dc-4701-aaf0-5e241ab82b74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678602 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/768859c6-079b-4947-ad85-d6bf78ac1023-hosts-file\") pod \"node-resolver-rjxhs\" (UID: \"768859c6-079b-4947-ad85-d6bf78ac1023\") " pod="openshift-dns/node-resolver-rjxhs" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678812 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.679937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678801 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-var-lib-cni-bin\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678834 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f9e4986-4f46-4007-b625-b49c6e766dec-env-overrides\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678870 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-run-ovn\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678874 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-hostroot\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678915 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-sysconfig\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678947 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f9e4986-4f46-4007-b625-b49c6e766dec-ovnkube-script-lib\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678955 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6964db66-5d84-43e3-8f76-e7e23aeb63d3-dbus\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678947 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.678997 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-run-systemd\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679009 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679012 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-sysconfig\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679028 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-var-lib-kubelet\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679041 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f9e4986-4f46-4007-b625-b49c6e766dec-ovn-node-metrics-cert\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679076 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-var-lib-kubelet\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679142 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-sys-fs\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679211 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-cni-netd\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679239 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-host\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.680807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679266 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6964db66-5d84-43e3-8f76-e7e23aeb63d3-kubelet-config\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679309 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-sysctl-d\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679320 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-cni-netd\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679333 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4412bdbc-bb2e-485e-bcf2-58166173a080-tmp\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679359 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-socket-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679391 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6964db66-5d84-43e3-8f76-e7e23aeb63d3-kubelet-config\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679407 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17977676-3dc1-41d5-8645-2a040631675b-iptables-alerter-script\") pod \"iptables-alerter-z84hz\" (UID: \"17977676-3dc1-41d5-8645-2a040631675b\") " pod="openshift-network-operator/iptables-alerter-z84hz" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679462 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-host\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679394 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-system-cni-dir\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679501 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-conf-dir\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679504 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-sysctl-d\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679526 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44c51362-b659-4fe7-9aed-1d649b89d193-socket-dir\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679526 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-daemon-config\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679595 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-etc-kubernetes\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679622 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4eb2215-40df-4faf-9040-0b4a38ed8198-konnectivity-ca\") pod \"konnectivity-agent-grpkq\" (UID: \"f4eb2215-40df-4faf-9040-0b4a38ed8198\") " pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679653 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-slash\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679680 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.682026 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679707 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5bp\" (UniqueName: \"kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp\") pod \"network-check-target-7cbth\" (UID: \"98ba8f25-7586-4345-8e2e-4c87629466ca\") " pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679767 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17977676-3dc1-41d5-8645-2a040631675b-host-slash\") pod \"iptables-alerter-z84hz\" (UID: \"17977676-3dc1-41d5-8645-2a040631675b\") " pod="openshift-network-operator/iptables-alerter-z84hz" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679778 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f9e4986-4f46-4007-b625-b49c6e766dec-host-slash\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679793 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-cni-dir\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679770 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b870da3-94e0-4301-8be9-17174f8a012a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679821 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-socket-dir-parent\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679851 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwbd4\" (UniqueName: \"kubernetes.io/projected/44c51362-b659-4fe7-9aed-1d649b89d193-kube-api-access-zwbd4\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679878 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-kubernetes\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679905 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/768859c6-079b-4947-ad85-d6bf78ac1023-tmp-dir\") pod \"node-resolver-rjxhs\" (UID: \"768859c6-079b-4947-ad85-d6bf78ac1023\") " pod="openshift-dns/node-resolver-rjxhs" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679930 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-cni-binary-copy\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679955 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-run-multus-certs\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.679981 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psn9h\" (UniqueName: \"kubernetes.io/projected/4412bdbc-bb2e-485e-bcf2-58166173a080-kube-api-access-psn9h\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.680008 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-os-release\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.680036 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s2ww\" (UniqueName: \"kubernetes.io/projected/6f9e4986-4f46-4007-b625-b49c6e766dec-kube-api-access-2s2ww\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.680054 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.680069 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-sysctl-conf\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.680099 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-systemd\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.682966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.680106 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17977676-3dc1-41d5-8645-2a040631675b-host-slash\") pod \"iptables-alerter-z84hz\" (UID: \"17977676-3dc1-41d5-8645-2a040631675b\") " pod="openshift-network-operator/iptables-alerter-z84hz" Jul 17 21:17:01.683722 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.680277 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-sysctl-conf\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.683722 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.680284 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-kubernetes\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.683722 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.680351 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-systemd\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.683722 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.680471 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b870da3-94e0-4301-8be9-17174f8a012a-os-release\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.683722 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.682735 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f9e4986-4f46-4007-b625-b49c6e766dec-ovn-node-metrics-cert\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.683722 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.682931 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4412bdbc-bb2e-485e-bcf2-58166173a080-tmp\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.690266 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.690242 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4412bdbc-bb2e-485e-bcf2-58166173a080-etc-tuned\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.702505 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.702479 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l49n7\" (UniqueName: \"kubernetes.io/projected/17977676-3dc1-41d5-8645-2a040631675b-kube-api-access-l49n7\") pod \"iptables-alerter-z84hz\" (UID: \"17977676-3dc1-41d5-8645-2a040631675b\") " pod="openshift-network-operator/iptables-alerter-z84hz" Jul 17 21:17:01.705750 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:01.705444 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 21:17:01.705750 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:01.705473 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 21:17:01.705750 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:01.705488 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dt5bp for pod openshift-network-diagnostics/network-check-target-7cbth: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:01.705750 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:01.705545 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp podName:98ba8f25-7586-4345-8e2e-4c87629466ca nodeName:}" failed. No retries permitted until 2026-07-17 21:17:02.205527002 +0000 UTC m=+2.974874975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dt5bp" (UniqueName: "kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp") pod "network-check-target-7cbth" (UID: "98ba8f25-7586-4345-8e2e-4c87629466ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:01.705750 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.705663 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwdnl\" (UniqueName: \"kubernetes.io/projected/7507ea3a-57dc-4701-aaf0-5e241ab82b74-kube-api-access-qwdnl\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:01.710447 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.709473 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx4bb\" (UniqueName: \"kubernetes.io/projected/82b932c6-788d-4a01-bdcb-c8c9b7303984-kube-api-access-gx4bb\") pod \"node-ca-6zztr\" (UID: \"82b932c6-788d-4a01-bdcb-c8c9b7303984\") " pod="openshift-image-registry/node-ca-6zztr" Jul 17 21:17:01.710447 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.709598 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s2ww\" (UniqueName: \"kubernetes.io/projected/6f9e4986-4f46-4007-b625-b49c6e766dec-kube-api-access-2s2ww\") pod \"ovnkube-node-vf7xx\" (UID: \"6f9e4986-4f46-4007-b625-b49c6e766dec\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.710447 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.709917 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psn9h\" (UniqueName: \"kubernetes.io/projected/4412bdbc-bb2e-485e-bcf2-58166173a080-kube-api-access-psn9h\") pod \"tuned-mndvj\" (UID: \"4412bdbc-bb2e-485e-bcf2-58166173a080\") " pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.710447 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.710291 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwbd4\" (UniqueName: \"kubernetes.io/projected/44c51362-b659-4fe7-9aed-1d649b89d193-kube-api-access-zwbd4\") pod \"aws-ebs-csi-driver-node-c4spm\" (UID: \"44c51362-b659-4fe7-9aed-1d649b89d193\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.711255 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.711235 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s8jp\" (UniqueName: \"kubernetes.io/projected/6b870da3-94e0-4301-8be9-17174f8a012a-kube-api-access-8s8jp\") pod \"multus-additional-cni-plugins-fkkg4\" (UID: \"6b870da3-94e0-4301-8be9-17174f8a012a\") " pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.719469 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.719370 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" event={"ID":"17a5673d75761b16d9c36066d01f1326","Type":"ContainerStarted","Data":"b22843456d3ccaf3890829db505150d43dbf7b69ab7289e652c7a5393db494a3"} Jul 17 21:17:01.720725 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.720648 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-188.ec2.internal" event={"ID":"fba93f9b97932f873e087f216da95d10","Type":"ContainerStarted","Data":"f2f01711578986b3b916a0c4d4afd5f8cfadd32790e718fee62c198fc297827c"} Jul 17 21:17:01.780577 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780530 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-system-cni-dir\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.780775 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780606 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-conf-dir\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.780775 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780636 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-daemon-config\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.780775 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780662 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-etc-kubernetes\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.780775 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780691 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4eb2215-40df-4faf-9040-0b4a38ed8198-konnectivity-ca\") pod \"konnectivity-agent-grpkq\" (UID: \"f4eb2215-40df-4faf-9040-0b4a38ed8198\") " pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:01.780775 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780696 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-conf-dir\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.780775 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780735 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-cni-dir\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.780775 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780754 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-etc-kubernetes\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.780775 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780760 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-socket-dir-parent\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780788 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/768859c6-079b-4947-ad85-d6bf78ac1023-tmp-dir\") pod \"node-resolver-rjxhs\" (UID: \"768859c6-079b-4947-ad85-d6bf78ac1023\") " pod="openshift-dns/node-resolver-rjxhs" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780814 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-cni-binary-copy\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780838 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-run-multus-certs\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780867 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-os-release\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780950 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-socket-dir-parent\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.780657 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-system-cni-dir\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781017 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4eb2215-40df-4faf-9040-0b4a38ed8198-agent-certs\") pod \"konnectivity-agent-grpkq\" (UID: \"f4eb2215-40df-4faf-9040-0b4a38ed8198\") " pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781055 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-cni-dir\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781093 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8sz9\" (UniqueName: \"kubernetes.io/projected/768859c6-079b-4947-ad85-d6bf78ac1023-kube-api-access-z8sz9\") pod \"node-resolver-rjxhs\" (UID: \"768859c6-079b-4947-ad85-d6bf78ac1023\") " pod="openshift-dns/node-resolver-rjxhs" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781118 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-run-multus-certs\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781159 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-run-k8s-cni-cncf-io\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781120 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-run-k8s-cni-cncf-io\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781217 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-run-netns\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781243 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-var-lib-cni-multus\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781266 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/768859c6-079b-4947-ad85-d6bf78ac1023-tmp-dir\") pod \"node-resolver-rjxhs\" (UID: \"768859c6-079b-4947-ad85-d6bf78ac1023\") " pod="openshift-dns/node-resolver-rjxhs" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781283 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-var-lib-cni-multus\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781292 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-os-release\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781295 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-run-netns\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781325 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8qzw\" (UniqueName: \"kubernetes.io/projected/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-kube-api-access-x8qzw\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781361 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-cnibin\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781402 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/768859c6-079b-4947-ad85-d6bf78ac1023-hosts-file\") pod \"node-resolver-rjxhs\" (UID: \"768859c6-079b-4947-ad85-d6bf78ac1023\") " pod="openshift-dns/node-resolver-rjxhs" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781431 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-var-lib-cni-bin\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781444 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-multus-daemon-config\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781457 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-hostroot\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781486 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-var-lib-kubelet\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781496 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-var-lib-cni-bin\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781538 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4eb2215-40df-4faf-9040-0b4a38ed8198-konnectivity-ca\") pod \"konnectivity-agent-grpkq\" (UID: \"f4eb2215-40df-4faf-9040-0b4a38ed8198\") " pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781577 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-host-var-lib-kubelet\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781592 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-hostroot\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781676 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/768859c6-079b-4947-ad85-d6bf78ac1023-hosts-file\") pod \"node-resolver-rjxhs\" (UID: \"768859c6-079b-4947-ad85-d6bf78ac1023\") " pod="openshift-dns/node-resolver-rjxhs" Jul 17 21:17:01.781884 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.781685 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-cnibin\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.782985 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.782164 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-cni-binary-copy\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.784846 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.784822 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4eb2215-40df-4faf-9040-0b4a38ed8198-agent-certs\") pod \"konnectivity-agent-grpkq\" (UID: \"f4eb2215-40df-4faf-9040-0b4a38ed8198\") " pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:01.795372 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.795314 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8sz9\" (UniqueName: \"kubernetes.io/projected/768859c6-079b-4947-ad85-d6bf78ac1023-kube-api-access-z8sz9\") pod \"node-resolver-rjxhs\" (UID: \"768859c6-079b-4947-ad85-d6bf78ac1023\") " pod="openshift-dns/node-resolver-rjxhs" Jul 17 21:17:01.795988 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.795961 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8qzw\" (UniqueName: \"kubernetes.io/projected/dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a-kube-api-access-x8qzw\") pod \"multus-57924\" (UID: \"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a\") " pod="openshift-multus/multus-57924" Jul 17 21:17:01.864505 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.864468 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z84hz" Jul 17 21:17:01.874625 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:01.874591 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17977676_3dc1_41d5_8645_2a040631675b.slice/crio-2db9f23b2416c4f3a9b4db33f6ba747fcd1bfc34023ac13963bf0b34fa403fcd WatchSource:0}: Error finding container 2db9f23b2416c4f3a9b4db33f6ba747fcd1bfc34023ac13963bf0b34fa403fcd: Status 404 returned error can't find the container with id 2db9f23b2416c4f3a9b4db33f6ba747fcd1bfc34023ac13963bf0b34fa403fcd Jul 17 21:17:01.881320 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.881293 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 21:17:01.885576 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.885558 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:01.893933 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.893903 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mndvj" Jul 17 21:17:01.903765 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.903746 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6zztr" Jul 17 21:17:01.910360 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.910340 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" Jul 17 21:17:01.928313 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.928284 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fkkg4" Jul 17 21:17:01.937162 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.937134 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rjxhs" Jul 17 21:17:01.948760 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.948739 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-57924" Jul 17 21:17:01.954530 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:01.954512 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:02.185933 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:02.185836 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:02.186096 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:02.185948 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:02.186096 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:02.185989 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:02.186096 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:02.186048 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:02.186096 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:02.186064 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret podName:6964db66-5d84-43e3-8f76-e7e23aeb63d3 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:03.186043525 +0000 UTC m=+3.955391503 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret") pod "global-pull-secret-syncer-79tgn" (UID: "6964db66-5d84-43e3-8f76-e7e23aeb63d3") : object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:02.186096 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:02.186091 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs podName:7507ea3a-57dc-4701-aaf0-5e241ab82b74 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:03.18607814 +0000 UTC m=+3.955426098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs") pod "network-metrics-daemon-6w8mk" (UID: "7507ea3a-57dc-4701-aaf0-5e241ab82b74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:02.286721 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:02.286684 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5bp\" (UniqueName: \"kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp\") pod \"network-check-target-7cbth\" (UID: \"98ba8f25-7586-4345-8e2e-4c87629466ca\") " pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:02.286882 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:02.286871 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 21:17:02.286945 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:02.286888 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 21:17:02.286945 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:02.286898 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dt5bp for pod openshift-network-diagnostics/network-check-target-7cbth: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:02.287034 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:02.286959 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp podName:98ba8f25-7586-4345-8e2e-4c87629466ca nodeName:}" failed. No retries permitted until 2026-07-17 21:17:03.286944637 +0000 UTC m=+4.056292595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dt5bp" (UniqueName: "kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp") pod "network-check-target-7cbth" (UID: "98ba8f25-7586-4345-8e2e-4c87629466ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:02.618646 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:02.618602 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 21:12:01 +0000 UTC" deadline="2027-12-15 01:25:34.756719508 +0000 UTC" Jul 17 21:17:02.618646 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:02.618641 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12364h8m32.138084712s" Jul 17 21:17:02.714676 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:02.714639 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:02.714838 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:02.714773 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:02.722832 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:02.722785 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z84hz" event={"ID":"17977676-3dc1-41d5-8645-2a040631675b","Type":"ContainerStarted","Data":"2db9f23b2416c4f3a9b4db33f6ba747fcd1bfc34023ac13963bf0b34fa403fcd"} Jul 17 21:17:02.941239 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:02.941188 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9e4986_4f46_4007_b625_b49c6e766dec.slice/crio-e0c4ef03419893bfab543f750f558ec9d44185ab1521c8fca76f27e208e027cc WatchSource:0}: Error finding container e0c4ef03419893bfab543f750f558ec9d44185ab1521c8fca76f27e208e027cc: Status 404 returned error can't find the container with id e0c4ef03419893bfab543f750f558ec9d44185ab1521c8fca76f27e208e027cc Jul 17 21:17:02.959875 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:02.959846 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b870da3_94e0_4301_8be9_17174f8a012a.slice/crio-30d6def5ed1b4bdbeaaff78b3cd091e5551db44554ff531b7d1e71b77178102b WatchSource:0}: Error finding container 30d6def5ed1b4bdbeaaff78b3cd091e5551db44554ff531b7d1e71b77178102b: Status 404 returned error can't find the container with id 30d6def5ed1b4bdbeaaff78b3cd091e5551db44554ff531b7d1e71b77178102b Jul 17 21:17:02.968707 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:02.968662 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbdbe4e0_e2fb_4591_93b3_6eaf6d7b9c8a.slice/crio-fe94845ba93ce6dc6331743a3f4c657d620a003d01df77e11b2d9218dad7a160 WatchSource:0}: Error finding container fe94845ba93ce6dc6331743a3f4c657d620a003d01df77e11b2d9218dad7a160: Status 404 returned error can't find the container with id fe94845ba93ce6dc6331743a3f4c657d620a003d01df77e11b2d9218dad7a160 Jul 17 21:17:02.969727 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:02.969649 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4412bdbc_bb2e_485e_bcf2_58166173a080.slice/crio-41aa97d3a17a349bfa83f8b421bd826f68265cb0f3889a129e78d5b8209ef1a9 WatchSource:0}: Error finding container 41aa97d3a17a349bfa83f8b421bd826f68265cb0f3889a129e78d5b8209ef1a9: Status 404 returned error can't find the container with id 41aa97d3a17a349bfa83f8b421bd826f68265cb0f3889a129e78d5b8209ef1a9 Jul 17 21:17:02.971640 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:02.970358 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c51362_b659_4fe7_9aed_1d649b89d193.slice/crio-3caa8a4ef4167be74fdc78deaaaff765a24a678f1771b5f93ff0faebe0a85aee WatchSource:0}: Error finding container 3caa8a4ef4167be74fdc78deaaaff765a24a678f1771b5f93ff0faebe0a85aee: Status 404 returned error can't find the container with id 3caa8a4ef4167be74fdc78deaaaff765a24a678f1771b5f93ff0faebe0a85aee Jul 17 21:17:02.972107 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:02.972080 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4eb2215_40df_4faf_9040_0b4a38ed8198.slice/crio-e1e6c89b6b558628a6520caa0409ec79947065cb082693309783c2637cc1c611 WatchSource:0}: Error finding container e1e6c89b6b558628a6520caa0409ec79947065cb082693309783c2637cc1c611: Status 404 returned error can't find the container with id e1e6c89b6b558628a6520caa0409ec79947065cb082693309783c2637cc1c611 Jul 17 21:17:02.972544 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:02.972518 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768859c6_079b_4947_ad85_d6bf78ac1023.slice/crio-be0be377eb421b5ca08e5d185f8035be06a4e7ebf29b279d5f07a3b6254cf33e WatchSource:0}: Error finding container be0be377eb421b5ca08e5d185f8035be06a4e7ebf29b279d5f07a3b6254cf33e: Status 404 returned error can't find the container with id be0be377eb421b5ca08e5d185f8035be06a4e7ebf29b279d5f07a3b6254cf33e Jul 17 21:17:02.973930 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:02.973742 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b932c6_788d_4a01_bdcb_c8c9b7303984.slice/crio-317ec90fcbc2b25900676c329c04fe791e82fbb498ea4744e334d27821efa57d WatchSource:0}: Error finding container 317ec90fcbc2b25900676c329c04fe791e82fbb498ea4744e334d27821efa57d: Status 404 returned error can't find the container with id 317ec90fcbc2b25900676c329c04fe791e82fbb498ea4744e334d27821efa57d Jul 17 21:17:03.195757 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.195494 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:03.195888 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.195795 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:03.195962 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:03.195946 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:03.196021 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:03.196009 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret podName:6964db66-5d84-43e3-8f76-e7e23aeb63d3 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:05.195989445 +0000 UTC m=+5.965337408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret") pod "global-pull-secret-syncer-79tgn" (UID: "6964db66-5d84-43e3-8f76-e7e23aeb63d3") : object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:03.196087 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:03.196069 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:03.196145 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:03.196101 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs podName:7507ea3a-57dc-4701-aaf0-5e241ab82b74 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:05.19608902 +0000 UTC m=+5.965436983 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs") pod "network-metrics-daemon-6w8mk" (UID: "7507ea3a-57dc-4701-aaf0-5e241ab82b74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:03.296831 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.296790 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5bp\" (UniqueName: \"kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp\") pod \"network-check-target-7cbth\" (UID: \"98ba8f25-7586-4345-8e2e-4c87629466ca\") " pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:03.297004 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:03.296993 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 21:17:03.297082 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:03.297017 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 21:17:03.297082 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:03.297032 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dt5bp for pod openshift-network-diagnostics/network-check-target-7cbth: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:03.297186 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:03.297098 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp podName:98ba8f25-7586-4345-8e2e-4c87629466ca nodeName:}" failed. No retries permitted until 2026-07-17 21:17:05.297078205 +0000 UTC m=+6.066426162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dt5bp" (UniqueName: "kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp") pod "network-check-target-7cbth" (UID: "98ba8f25-7586-4345-8e2e-4c87629466ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:03.619606 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.619487 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 21:12:01 +0000 UTC" deadline="2028-04-15 15:16:25.60762246 +0000 UTC" Jul 17 21:17:03.619606 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.619571 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15305h59m21.988056737s" Jul 17 21:17:03.716989 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.716485 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:03.716989 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:03.716590 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:03.716989 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.716696 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:03.716989 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:03.716793 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:03.735428 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.735393 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" event={"ID":"44c51362-b659-4fe7-9aed-1d649b89d193","Type":"ContainerStarted","Data":"3caa8a4ef4167be74fdc78deaaaff765a24a678f1771b5f93ff0faebe0a85aee"} Jul 17 21:17:03.738784 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.738755 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6zztr" event={"ID":"82b932c6-788d-4a01-bdcb-c8c9b7303984","Type":"ContainerStarted","Data":"317ec90fcbc2b25900676c329c04fe791e82fbb498ea4744e334d27821efa57d"} Jul 17 21:17:03.741939 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.741903 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-57924" event={"ID":"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a","Type":"ContainerStarted","Data":"fe94845ba93ce6dc6331743a3f4c657d620a003d01df77e11b2d9218dad7a160"} Jul 17 21:17:03.755410 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.754730 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-188.ec2.internal" event={"ID":"fba93f9b97932f873e087f216da95d10","Type":"ContainerStarted","Data":"d703b17b765fe52a20ff49c95a4e481b74ae12cd477a03683c1c7bf19282c42f"} Jul 17 21:17:03.758141 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.758056 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-grpkq" event={"ID":"f4eb2215-40df-4faf-9040-0b4a38ed8198","Type":"ContainerStarted","Data":"e1e6c89b6b558628a6520caa0409ec79947065cb082693309783c2637cc1c611"} Jul 17 21:17:03.770485 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.770415 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rjxhs" event={"ID":"768859c6-079b-4947-ad85-d6bf78ac1023","Type":"ContainerStarted","Data":"be0be377eb421b5ca08e5d185f8035be06a4e7ebf29b279d5f07a3b6254cf33e"} Jul 17 21:17:03.775321 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.775288 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mndvj" event={"ID":"4412bdbc-bb2e-485e-bcf2-58166173a080","Type":"ContainerStarted","Data":"41aa97d3a17a349bfa83f8b421bd826f68265cb0f3889a129e78d5b8209ef1a9"} Jul 17 21:17:03.778206 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.778170 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkkg4" event={"ID":"6b870da3-94e0-4301-8be9-17174f8a012a","Type":"ContainerStarted","Data":"30d6def5ed1b4bdbeaaff78b3cd091e5551db44554ff531b7d1e71b77178102b"} Jul 17 21:17:03.781886 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.781856 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" event={"ID":"6f9e4986-4f46-4007-b625-b49c6e766dec","Type":"ContainerStarted","Data":"e0c4ef03419893bfab543f750f558ec9d44185ab1521c8fca76f27e208e027cc"} Jul 17 21:17:03.789069 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.789041 2554 generic.go:356] "Generic (PLEG): container finished" podID="17a5673d75761b16d9c36066d01f1326" containerID="c965e290947391141474d1a2a04ee4a567d1ae8d0636636df6f253c45c19b2cb" exitCode=0 Jul 17 21:17:03.789191 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.789081 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" event={"ID":"17a5673d75761b16d9c36066d01f1326","Type":"ContainerDied","Data":"c965e290947391141474d1a2a04ee4a567d1ae8d0636636df6f253c45c19b2cb"} Jul 17 21:17:03.807681 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:03.807603 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-188.ec2.internal" podStartSLOduration=2.807581464 podStartE2EDuration="2.807581464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:03.772951116 +0000 UTC m=+4.542299097" watchObservedRunningTime="2026-07-17 21:17:03.807581464 +0000 UTC m=+4.576929449" Jul 17 21:17:04.713641 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:04.713589 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:04.714134 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:04.713737 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:04.805643 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:04.805590 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" event={"ID":"17a5673d75761b16d9c36066d01f1326","Type":"ContainerStarted","Data":"ce6c359812655428ca3d300c3d8a3c36b17ff388c73270635887ebdb48462a32"} Jul 17 21:17:05.213660 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:05.213620 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:05.213840 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:05.213687 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:05.213840 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:05.213820 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:05.213914 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:05.213883 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret podName:6964db66-5d84-43e3-8f76-e7e23aeb63d3 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:09.213865059 +0000 UTC m=+9.983213019 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret") pod "global-pull-secret-syncer-79tgn" (UID: "6964db66-5d84-43e3-8f76-e7e23aeb63d3") : object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:05.214301 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:05.214282 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:05.214364 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:05.214335 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs podName:7507ea3a-57dc-4701-aaf0-5e241ab82b74 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:09.214320437 +0000 UTC m=+9.983668397 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs") pod "network-metrics-daemon-6w8mk" (UID: "7507ea3a-57dc-4701-aaf0-5e241ab82b74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:05.314296 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:05.314214 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5bp\" (UniqueName: \"kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp\") pod \"network-check-target-7cbth\" (UID: \"98ba8f25-7586-4345-8e2e-4c87629466ca\") " pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:05.314474 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:05.314429 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 21:17:05.314474 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:05.314447 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 21:17:05.314474 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:05.314460 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dt5bp for pod openshift-network-diagnostics/network-check-target-7cbth: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:05.314603 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:05.314521 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp podName:98ba8f25-7586-4345-8e2e-4c87629466ca nodeName:}" failed. No retries permitted until 2026-07-17 21:17:09.314503256 +0000 UTC m=+10.083851235 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dt5bp" (UniqueName: "kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp") pod "network-check-target-7cbth" (UID: "98ba8f25-7586-4345-8e2e-4c87629466ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:05.721969 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:05.721246 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:05.721969 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:05.721382 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:05.721969 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:05.721794 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:05.721969 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:05.721916 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:06.714445 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:06.714403 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:06.714674 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:06.714532 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:07.715689 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:07.715656 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:07.716124 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:07.715812 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:07.716124 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:07.715858 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:07.716124 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:07.715975 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:08.713715 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:08.713678 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:08.713895 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:08.713808 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:09.250454 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:09.250420 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:09.250935 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:09.250478 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:09.250935 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:09.250597 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:09.250935 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:09.250610 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:09.250935 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:09.250669 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs podName:7507ea3a-57dc-4701-aaf0-5e241ab82b74 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:17.250649425 +0000 UTC m=+18.019997394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs") pod "network-metrics-daemon-6w8mk" (UID: "7507ea3a-57dc-4701-aaf0-5e241ab82b74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:09.250935 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:09.250689 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret podName:6964db66-5d84-43e3-8f76-e7e23aeb63d3 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:17.25067893 +0000 UTC m=+18.020026891 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret") pod "global-pull-secret-syncer-79tgn" (UID: "6964db66-5d84-43e3-8f76-e7e23aeb63d3") : object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:09.351814 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:09.351778 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5bp\" (UniqueName: \"kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp\") pod \"network-check-target-7cbth\" (UID: \"98ba8f25-7586-4345-8e2e-4c87629466ca\") " pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:09.352017 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:09.351993 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 21:17:09.352017 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:09.352022 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 21:17:09.352191 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:09.352035 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dt5bp for pod openshift-network-diagnostics/network-check-target-7cbth: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:09.352191 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:09.352093 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp podName:98ba8f25-7586-4345-8e2e-4c87629466ca nodeName:}" failed. No retries permitted until 2026-07-17 21:17:17.352073444 +0000 UTC m=+18.121421422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dt5bp" (UniqueName: "kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp") pod "network-check-target-7cbth" (UID: "98ba8f25-7586-4345-8e2e-4c87629466ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:09.723669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:09.723638 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:09.723847 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:09.723834 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:09.723980 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:09.723956 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:09.724245 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:09.724206 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:10.714444 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:10.714416 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:10.714922 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:10.714564 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:11.715191 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:11.714570 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:11.715191 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:11.714695 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:11.715191 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:11.715044 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:11.715191 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:11.715146 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:12.714412 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:12.714378 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:12.714745 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:12.714507 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:13.714680 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:13.714385 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:13.714680 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:13.714390 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:13.714680 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:13.714523 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:13.714680 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:13.714611 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:14.713564 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:14.713508 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:14.713728 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:14.713645 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:15.713999 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:15.713958 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:15.714408 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:15.714112 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:15.714408 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:15.714168 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:15.714408 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:15.714289 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:16.714349 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:16.714318 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:16.714825 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:16.714434 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:17.314786 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:17.314738 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:17.314969 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:17.314803 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:17.314969 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:17.314931 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:17.315080 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:17.314993 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret podName:6964db66-5d84-43e3-8f76-e7e23aeb63d3 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:33.314978288 +0000 UTC m=+34.084326264 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret") pod "global-pull-secret-syncer-79tgn" (UID: "6964db66-5d84-43e3-8f76-e7e23aeb63d3") : object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:17.315315 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:17.315284 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:17.315429 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:17.315355 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs podName:7507ea3a-57dc-4701-aaf0-5e241ab82b74 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:33.315337333 +0000 UTC m=+34.084685302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs") pod "network-metrics-daemon-6w8mk" (UID: "7507ea3a-57dc-4701-aaf0-5e241ab82b74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:17.415313 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:17.415272 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5bp\" (UniqueName: \"kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp\") pod \"network-check-target-7cbth\" (UID: \"98ba8f25-7586-4345-8e2e-4c87629466ca\") " pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:17.415491 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:17.415422 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 21:17:17.415491 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:17.415452 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 21:17:17.415491 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:17.415464 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dt5bp for pod openshift-network-diagnostics/network-check-target-7cbth: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:17.415669 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:17.415532 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp podName:98ba8f25-7586-4345-8e2e-4c87629466ca nodeName:}" failed. No retries permitted until 2026-07-17 21:17:33.415513332 +0000 UTC m=+34.184861302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dt5bp" (UniqueName: "kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp") pod "network-check-target-7cbth" (UID: "98ba8f25-7586-4345-8e2e-4c87629466ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:17.714711 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:17.714627 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:17.714711 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:17.714657 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:17.715119 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:17.714780 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:17.715119 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:17.714920 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:18.714036 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:18.713991 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:18.714208 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:18.714107 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:19.714676 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:19.714637 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:19.715106 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:19.714760 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:19.715106 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:19.714826 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:19.715106 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:19.714944 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:20.713557 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:20.713505 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:20.713730 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:20.713649 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:21.717337 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.715921 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:21.717337 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:21.716215 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:21.717337 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.716298 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:21.717337 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:21.716384 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:21.843334 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.843148 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" event={"ID":"44c51362-b659-4fe7-9aed-1d649b89d193","Type":"ContainerStarted","Data":"a1d205a11e0e686c36deea25076e10e347fefb81505b8ec89ef186b7e075234c"} Jul 17 21:17:21.844657 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.844634 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6zztr" event={"ID":"82b932c6-788d-4a01-bdcb-c8c9b7303984","Type":"ContainerStarted","Data":"c5f8493e6d528a7a5868270c6339b37dd42ebb6cc230b4be42abeff6ebd8afba"} Jul 17 21:17:21.846024 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.845998 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-57924" event={"ID":"dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a","Type":"ContainerStarted","Data":"f74125a679dc90c3e08a0bd3efe80896c3ce52e599e1d1ce68c742e1907c981b"} Jul 17 21:17:21.847440 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.847414 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-grpkq" event={"ID":"f4eb2215-40df-4faf-9040-0b4a38ed8198","Type":"ContainerStarted","Data":"e0fef395a49ffcd296c2702c7e6924c92ebffe46723108b80077e931b83936b8"} Jul 17 21:17:21.848836 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.848812 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rjxhs" event={"ID":"768859c6-079b-4947-ad85-d6bf78ac1023","Type":"ContainerStarted","Data":"bcb4b8dad56c123e342bff01ec028400b670e6ab2281e2ea87324d956b0e6093"} Jul 17 21:17:21.850219 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.850197 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mndvj" event={"ID":"4412bdbc-bb2e-485e-bcf2-58166173a080","Type":"ContainerStarted","Data":"931512fe70d449ffb80ca55f34dc54844ec36cf6d0ca87ada89f8d9b6ca2c170"} Jul 17 21:17:21.851461 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.851442 2554 generic.go:356] "Generic (PLEG): container finished" podID="6b870da3-94e0-4301-8be9-17174f8a012a" containerID="685f1d80cba85ab067f485146fa5006b06cfad6170c15818b4f84537185bd0a3" exitCode=0 Jul 17 21:17:21.851562 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.851511 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkkg4" event={"ID":"6b870da3-94e0-4301-8be9-17174f8a012a","Type":"ContainerDied","Data":"685f1d80cba85ab067f485146fa5006b06cfad6170c15818b4f84537185bd0a3"} Jul 17 21:17:21.853673 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.853650 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" event={"ID":"6f9e4986-4f46-4007-b625-b49c6e766dec","Type":"ContainerStarted","Data":"d5d3e2623a0d1e20b7a2ce6c4fc9d58ca3d72ea20660923b331956433dcb56f3"} Jul 17 21:17:21.853746 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.853732 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" event={"ID":"6f9e4986-4f46-4007-b625-b49c6e766dec","Type":"ContainerStarted","Data":"50f63ce09f76840d1911817b3ad2bf104102cbdada1271f1be352450bc785bc3"} Jul 17 21:17:21.853803 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.853790 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" event={"ID":"6f9e4986-4f46-4007-b625-b49c6e766dec","Type":"ContainerStarted","Data":"a31773a79f9f278ad5fb7ebccc258b3ce4a5ce1ae4aeecb31b28a66251bc980e"} Jul 17 21:17:21.863208 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.863172 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6zztr" podStartSLOduration=3.361416968 podStartE2EDuration="21.863159197s" totalImagesPullingTime="18.501742229s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:21.862496305 +0000 UTC m=+22.631844283" watchObservedRunningTime="2026-07-17 21:17:21.863159197 +0000 UTC m=+22.632507176" Jul 17 21:17:21.863300 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.863236 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-188.ec2.internal" podStartSLOduration=20.863233181 podStartE2EDuration="20.863233181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:04.826343613 +0000 UTC m=+5.595691601" watchObservedRunningTime="2026-07-17 21:17:21.863233181 +0000 UTC m=+22.632581158" Jul 17 21:17:21.881638 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.881519 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mndvj" podStartSLOduration=3.373897646 podStartE2EDuration="21.881504925s" totalImagesPullingTime="18.507607279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:21.88085521 +0000 UTC m=+22.650203188" watchObservedRunningTime="2026-07-17 21:17:21.881504925 +0000 UTC m=+22.650852901" Jul 17 21:17:21.944480 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.944426 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rjxhs" podStartSLOduration=3.470300062 podStartE2EDuration="21.944406207s" totalImagesPullingTime="18.474106145s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:21.943770635 +0000 UTC m=+22.713118614" watchObservedRunningTime="2026-07-17 21:17:21.944406207 +0000 UTC m=+22.713754186" Jul 17 21:17:21.973880 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.973841 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-57924" podStartSLOduration=3.439631115 podStartE2EDuration="21.973823834s" totalImagesPullingTime="18.534192719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:21.973390651 +0000 UTC m=+22.742738630" watchObservedRunningTime="2026-07-17 21:17:21.973823834 +0000 UTC m=+22.743171823" Jul 17 21:17:21.992286 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:21.992242 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-grpkq" podStartSLOduration=8.079513138 podStartE2EDuration="21.992229144s" totalImagesPullingTime="13.912716006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:21.991809552 +0000 UTC m=+22.761157531" watchObservedRunningTime="2026-07-17 21:17:21.992229144 +0000 UTC m=+22.761577121" Jul 17 21:17:22.713785 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:22.713756 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:22.713906 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:22.713885 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:22.786648 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:22.786621 2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 17 21:17:22.859606 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:22.859571 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" event={"ID":"6f9e4986-4f46-4007-b625-b49c6e766dec","Type":"ContainerStarted","Data":"25f55c31cff825876a2db09ec0a215df5fdecac1738cd427ec197072b8d0f351"} Jul 17 21:17:22.859786 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:22.859672 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" event={"ID":"6f9e4986-4f46-4007-b625-b49c6e766dec","Type":"ContainerStarted","Data":"420a5915685553acc123ee060a5e9c3b612c532c1c90b92d7cd56fa9ee15cde9"} Jul 17 21:17:22.859786 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:22.859715 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" event={"ID":"6f9e4986-4f46-4007-b625-b49c6e766dec","Type":"ContainerStarted","Data":"9546fbd7e3dec640f446db6250def813ee9fc06c55db8babf6ed85844e2fdb91"} Jul 17 21:17:22.861534 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:22.861505 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" event={"ID":"44c51362-b659-4fe7-9aed-1d649b89d193","Type":"ContainerStarted","Data":"dbf25bed63fc0335ec6903f3e5cdf08c49e323d1afd5cf17f90600719d389565"} Jul 17 21:17:23.632934 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:23.632783 2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T21:17:22.786643483Z","UUID":"66a01daf-a495-4c4b-ad47-b731207e99b3","Handler":null,"Name":"","Endpoint":""} Jul 17 21:17:23.634305 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:23.634281 2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 17 21:17:23.634494 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:23.634322 2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 17 21:17:23.651260 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:23.651234 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:23.651816 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:23.651799 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:23.714119 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:23.714091 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:23.714321 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:23.714091 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:23.714321 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:23.714231 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:23.714321 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:23.714291 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:23.866659 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:23.866466 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" event={"ID":"44c51362-b659-4fe7-9aed-1d649b89d193","Type":"ContainerStarted","Data":"d5f7a536b81e62afcb41cd5dace815c9bc115e33dbf35c8dc05b548813ba1d98"} Jul 17 21:17:23.868491 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:23.868156 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z84hz" event={"ID":"17977676-3dc1-41d5-8645-2a040631675b","Type":"ContainerStarted","Data":"884b5accc1ce8ce701bd50a1778e86e89ed64fdc04252d76ea6c8e8c81f84251"} Jul 17 21:17:23.888422 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:23.888344 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4spm" podStartSLOduration=3.52581824 podStartE2EDuration="23.888330311s" totalImagesPullingTime="20.362512071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:23.888220339 +0000 UTC m=+24.657568317" watchObservedRunningTime="2026-07-17 21:17:23.888330311 +0000 UTC m=+24.657678290" Jul 17 21:17:23.904659 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:23.904621 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z84hz" podStartSLOduration=4.595910989 podStartE2EDuration="23.904605073s" totalImagesPullingTime="19.308694084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:23.903876716 +0000 UTC m=+24.673224694" watchObservedRunningTime="2026-07-17 21:17:23.904605073 +0000 UTC m=+24.673953051" Jul 17 21:17:24.714251 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:24.714052 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:24.714400 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:24.714339 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:24.873846 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:24.873804 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" event={"ID":"6f9e4986-4f46-4007-b625-b49c6e766dec","Type":"ContainerStarted","Data":"9f10d4934a2cb7aa940682f3131c45f8feb82dccbe246500671b091b8871a851"} Jul 17 21:17:24.874262 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:24.873818 2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 17 21:17:25.714433 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:25.714396 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:25.714648 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:25.714560 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:25.714648 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:25.714574 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:25.714754 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:25.714650 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:26.713667 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:26.713647 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:26.714004 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:26.713736 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:26.888060 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:26.887880 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" event={"ID":"6f9e4986-4f46-4007-b625-b49c6e766dec","Type":"ContainerStarted","Data":"ed859de7ea5a44f85e5f107443796667bf4a6d3d24dcd8fcb209466d58a663f6"} Jul 17 21:17:26.888173 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:26.888160 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:26.888237 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:26.888217 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:26.888285 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:26.888261 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:26.906852 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:26.906828 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:26.907634 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:26.907609 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:26.920807 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:26.920764 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" podStartSLOduration=8.328743018 podStartE2EDuration="26.920751059s" totalImagesPullingTime="18.592008041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:26.920410291 +0000 UTC m=+27.689758268" watchObservedRunningTime="2026-07-17 21:17:26.920751059 +0000 UTC m=+27.690099048" Jul 17 21:17:27.714209 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:27.714174 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:27.714661 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:27.714174 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:27.714661 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:27.714301 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:27.714661 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:27.714350 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:27.891322 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:27.891243 2554 generic.go:356] "Generic (PLEG): container finished" podID="6b870da3-94e0-4301-8be9-17174f8a012a" containerID="fcff16e52d41a83028ad7fbe6e563e36e4b37b40ad05ac83cba5aee1e9be2ab4" exitCode=0 Jul 17 21:17:27.891464 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:27.891327 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkkg4" event={"ID":"6b870da3-94e0-4301-8be9-17174f8a012a","Type":"ContainerDied","Data":"fcff16e52d41a83028ad7fbe6e563e36e4b37b40ad05ac83cba5aee1e9be2ab4"} Jul 17 21:17:28.713598 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:28.713573 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:28.713768 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:28.713693 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:28.963036 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:28.962996 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6w8mk"] Jul 17 21:17:28.963469 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:28.963158 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:28.963469 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:28.963295 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:28.965927 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:28.965867 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7cbth"] Jul 17 21:17:28.966061 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:28.965970 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:28.966114 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:28.966055 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:28.980397 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:28.980371 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-79tgn"] Jul 17 21:17:28.980495 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:28.980453 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:28.980564 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:28.980527 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:29.897197 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:29.897162 2554 generic.go:356] "Generic (PLEG): container finished" podID="6b870da3-94e0-4301-8be9-17174f8a012a" containerID="417fb7192238f972419b2a464019bf2a5020688e57d395304597f63acf88bdbb" exitCode=0 Jul 17 21:17:29.897339 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:29.897220 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkkg4" event={"ID":"6b870da3-94e0-4301-8be9-17174f8a012a","Type":"ContainerDied","Data":"417fb7192238f972419b2a464019bf2a5020688e57d395304597f63acf88bdbb"} Jul 17 21:17:30.558500 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:30.558411 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:30.558899 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:30.558585 2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 17 21:17:30.559589 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:30.559566 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-grpkq" Jul 17 21:17:30.713677 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:30.713505 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:30.713811 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:30.713561 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:30.713811 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:30.713753 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:30.713896 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:30.713570 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:30.713896 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:30.713815 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:30.713896 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:30.713875 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:31.903688 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:31.903651 2554 generic.go:356] "Generic (PLEG): container finished" podID="6b870da3-94e0-4301-8be9-17174f8a012a" containerID="ea45a79784451d06e28ddd55aa3c5b90970b8085266ec293001d00443865c1a2" exitCode=0 Jul 17 21:17:31.903688 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:31.903688 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkkg4" event={"ID":"6b870da3-94e0-4301-8be9-17174f8a012a","Type":"ContainerDied","Data":"ea45a79784451d06e28ddd55aa3c5b90970b8085266ec293001d00443865c1a2"} Jul 17 21:17:32.713856 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:32.713816 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:32.714456 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:32.713856 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:32.714456 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:32.713854 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:32.714456 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:32.713967 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6w8mk" podUID="7507ea3a-57dc-4701-aaf0-5e241ab82b74" Jul 17 21:17:32.714456 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:32.714094 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cbth" podUID="98ba8f25-7586-4345-8e2e-4c87629466ca" Jul 17 21:17:32.714456 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:32.714252 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79tgn" podUID="6964db66-5d84-43e3-8f76-e7e23aeb63d3" Jul 17 21:17:33.324471 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:33.324426 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:33.325105 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:33.324483 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:33.325105 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:33.324584 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:33.325105 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:33.324641 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:33.325105 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:33.324655 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs podName:7507ea3a-57dc-4701-aaf0-5e241ab82b74 nodeName:}" failed. No retries permitted until 2026-07-17 21:18:05.324633736 +0000 UTC m=+66.093981717 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs") pod "network-metrics-daemon-6w8mk" (UID: "7507ea3a-57dc-4701-aaf0-5e241ab82b74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 21:17:33.325105 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:33.324698 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret podName:6964db66-5d84-43e3-8f76-e7e23aeb63d3 nodeName:}" failed. No retries permitted until 2026-07-17 21:18:05.324680108 +0000 UTC m=+66.094028071 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret") pod "global-pull-secret-syncer-79tgn" (UID: "6964db66-5d84-43e3-8f76-e7e23aeb63d3") : object "kube-system"/"original-pull-secret" not registered Jul 17 21:17:33.425343 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:33.425307 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5bp\" (UniqueName: \"kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp\") pod \"network-check-target-7cbth\" (UID: \"98ba8f25-7586-4345-8e2e-4c87629466ca\") " pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:33.425602 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:33.425440 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 21:17:33.425602 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:33.425460 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 21:17:33.425602 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:33.425471 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dt5bp for pod openshift-network-diagnostics/network-check-target-7cbth: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:33.425602 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:33.425527 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp podName:98ba8f25-7586-4345-8e2e-4c87629466ca nodeName:}" failed. No retries permitted until 2026-07-17 21:18:05.425513349 +0000 UTC m=+66.194861306 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dt5bp" (UniqueName: "kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp") pod "network-check-target-7cbth" (UID: "98ba8f25-7586-4345-8e2e-4c87629466ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 21:17:34.034776 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.034745 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-188.ec2.internal" event="NodeReady" Jul 17 21:17:34.034917 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.034900 2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 17 21:17:34.074974 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.074947 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69f9779877-887qk"] Jul 17 21:17:34.095063 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.095041 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42"] Jul 17 21:17:34.095222 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.095186 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.099416 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.099010 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 17 21:17:34.099416 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.099348 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fr9hx\"" Jul 17 21:17:34.099672 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.099654 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 17 21:17:34.100065 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.100045 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 17 21:17:34.106750 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.106132 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 17 21:17:34.110376 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.110357 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj"] Jul 17 21:17:34.110487 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.110472 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" Jul 17 21:17:34.113109 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.113087 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 17 21:17:34.113210 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.113119 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 17 21:17:34.113210 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.113155 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rgrbv\"" Jul 17 21:17:34.113210 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.113186 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 17 21:17:34.113581 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.113562 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 17 21:17:34.122723 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.122685 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ppljx"] Jul 17 21:17:34.122862 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.122838 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" Jul 17 21:17:34.127286 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.127269 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 17 21:17:34.127385 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.127321 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 17 21:17:34.127618 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.127603 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 17 21:17:34.127699 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.127652 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7zpf6\"" Jul 17 21:17:34.127757 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.127653 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 17 21:17:34.136822 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.136806 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74bdb64d9b-rxs6z"] Jul 17 21:17:34.137036 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.137018 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ppljx" Jul 17 21:17:34.140882 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.139901 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 17 21:17:34.140882 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.140162 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 17 21:17:34.140882 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.140420 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-l27fz\"" Jul 17 21:17:34.150775 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.150756 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm"] Jul 17 21:17:34.150869 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.150856 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.153437 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.153419 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 17 21:17:34.153654 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.153632 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 17 21:17:34.153945 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.153918 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 17 21:17:34.153945 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.153935 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 17 21:17:34.154105 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.153959 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 17 21:17:34.154286 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.154218 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 17 21:17:34.154286 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.154279 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j2lmg\"" Jul 17 21:17:34.165038 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.165006 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-k9wjf"] Jul 17 21:17:34.165235 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.165124 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:17:34.167580 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.167558 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wsskm\"" Jul 17 21:17:34.167951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.167801 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 17 21:17:34.167951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.167808 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 17 21:17:34.167951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.167862 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 17 21:17:34.179340 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.179316 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69f9779877-887qk"] Jul 17 21:17:34.179445 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.179355 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42"] Jul 17 21:17:34.179445 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.179375 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj"] Jul 17 21:17:34.179445 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.179395 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm"] Jul 17 21:17:34.179445 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.179417 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-k9wjf"] Jul 17 21:17:34.179445 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.179433 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74bdb64d9b-rxs6z"] Jul 17 21:17:34.179706 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.179450 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.179706 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.179452 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rvqhc"] Jul 17 21:17:34.181983 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.181966 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-jk9d2\"" Jul 17 21:17:34.182086 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.181986 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 17 21:17:34.182086 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.181964 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 17 21:17:34.182192 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.182134 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 17 21:17:34.182330 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.182312 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 17 21:17:34.188790 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.188771 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 17 21:17:34.203727 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.203701 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-qdvms"] Jul 17 21:17:34.203917 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.203838 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.206494 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.206439 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lwcbh\"" Jul 17 21:17:34.206494 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.206473 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 17 21:17:34.206666 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.206477 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 17 21:17:34.218190 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.218126 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v"] Jul 17 21:17:34.218297 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.218278 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.221506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.221485 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 17 21:17:34.221876 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.221860 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 17 21:17:34.222042 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.222028 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-75hkj\"" Jul 17 21:17:34.222277 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.222262 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 17 21:17:34.222472 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.222454 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 17 21:17:34.228518 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.228500 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 17 21:17:34.231979 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.231960 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.232077 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.231987 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-certificates\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.232077 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232004 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-jt7ll"] Jul 17 21:17:34.232077 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232038 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z89g\" (UniqueName: \"kubernetes.io/projected/133e55a6-f3de-428f-a9c6-fa697726d63a-kube-api-access-9z89g\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.232077 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232065 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-image-registry-private-configuration\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.232272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232084 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpkh7\" (UniqueName: \"kubernetes.io/projected/036870c8-94b9-4906-8b4a-ea70695c9c79-kube-api-access-kpkh7\") pod \"kube-storage-version-migrator-operator-7658d9b749-w8cbj\" (UID: \"036870c8-94b9-4906-8b4a-ea70695c9c79\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" Jul 17 21:17:34.232272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232108 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97cee0f9-881c-44b3-9c70-8dc81c28699b-ca-trust-extracted\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.232272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232134 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-bound-sa-token\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.232272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232163 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.232272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232188 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/036870c8-94b9-4906-8b4a-ea70695c9c79-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-w8cbj\" (UID: \"036870c8-94b9-4906-8b4a-ea70695c9c79\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" Jul 17 21:17:34.232272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232210 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/036870c8-94b9-4906-8b4a-ea70695c9c79-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-w8cbj\" (UID: \"036870c8-94b9-4906-8b4a-ea70695c9c79\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" Jul 17 21:17:34.232272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232232 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-trusted-ca\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.232272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232259 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fbe62ded-80ca-4471-ae23-dc1abdce5895-config\") pod \"service-ca-operator-5bcbc94c78-qhb42\" (UID: \"fbe62ded-80ca-4471-ae23-dc1abdce5895\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" Jul 17 21:17:34.232669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232290 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlcpd\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-kube-api-access-vlcpd\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.232669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232311 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:34.232669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232313 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fbe62ded-80ca-4471-ae23-dc1abdce5895-serving-cert\") pod \"service-ca-operator-5bcbc94c78-qhb42\" (UID: \"fbe62ded-80ca-4471-ae23-dc1abdce5895\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" Jul 17 21:17:34.232669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232387 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-default-certificate\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.232669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232418 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.232669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232451 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r598k\" (UniqueName: \"kubernetes.io/projected/fbe62ded-80ca-4471-ae23-dc1abdce5895-kube-api-access-r598k\") pod \"service-ca-operator-5bcbc94c78-qhb42\" (UID: \"fbe62ded-80ca-4471-ae23-dc1abdce5895\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" Jul 17 21:17:34.232669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232482 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmkw4\" (UniqueName: \"kubernetes.io/projected/389d5d02-d0be-4eff-bdea-bb0c695ca377-kube-api-access-nmkw4\") pod \"volume-data-source-validator-78cf4c6f6-ppljx\" (UID: \"389d5d02-d0be-4eff-bdea-bb0c695ca377\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ppljx" Jul 17 21:17:34.232669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232520 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-installation-pull-secrets\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.232669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.232577 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-stats-auth\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.236246 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.236193 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 17 21:17:34.237175 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.237154 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mjxnm\"" Jul 17 21:17:34.240919 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.237792 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 17 21:17:34.240919 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.238651 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 17 21:17:34.241629 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.241585 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 17 21:17:34.250223 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.250204 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hfktc"] Jul 17 21:17:34.250378 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.250357 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jt7ll" Jul 17 21:17:34.253389 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.253373 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tzbtb\"" Jul 17 21:17:34.253495 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.253452 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 21:17:34.253495 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.253474 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 21:17:34.274401 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.274382 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ppljx"] Jul 17 21:17:34.274514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.274424 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-qdvms"] Jul 17 21:17:34.274514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.274445 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rvqhc"] Jul 17 21:17:34.274514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.274468 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hfktc"] Jul 17 21:17:34.274514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.274487 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v"] Jul 17 21:17:34.274514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.274491 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hfktc" Jul 17 21:17:34.274514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.274509 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-jt7ll"] Jul 17 21:17:34.277558 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.277337 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 17 21:17:34.277558 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.277384 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 17 21:17:34.277558 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.277385 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7vktg\"" Jul 17 21:17:34.277558 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.277420 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 17 21:17:34.288006 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.287936 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:34.333201 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333167 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-image-registry-private-configuration\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.333797 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333210 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlcpd\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-kube-api-access-vlcpd\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.333797 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333340 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cefc4af-8f20-4aad-ba30-041f244f9740-config\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.333797 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333421 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.333797 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333451 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/036870c8-94b9-4906-8b4a-ea70695c9c79-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-w8cbj\" (UID: \"036870c8-94b9-4906-8b4a-ea70695c9c79\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" Jul 17 21:17:34.333797 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333480 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/999ecc4f-66b1-4ecb-97c5-c54040757d19-serving-cert\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.333797 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333505 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm84l\" (UniqueName: \"kubernetes.io/projected/a491e347-ad20-4da2-97bb-ce0d561846bb-kube-api-access-tm84l\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:34.333797 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333528 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fbe62ded-80ca-4471-ae23-dc1abdce5895-serving-cert\") pod \"service-ca-operator-5bcbc94c78-qhb42\" (UID: \"fbe62ded-80ca-4471-ae23-dc1abdce5895\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" Jul 17 21:17:34.333797 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333572 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:17:34.333797 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.333597 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:17:34.833574849 +0000 UTC m=+35.602922872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : configmap references non-existent config key: service-ca.crt Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333831 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333874 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333915 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-installation-pull-secrets\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333942 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.333964 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.333968 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c76d2b-6ab9-421d-97af-7c9e03181c7c-tmp-dir\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.333978 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f9779877-887qk: secret "image-registry-tls" not found Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334017 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-bound-sa-token\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.334035 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls podName:97cee0f9-881c-44b3-9c70-8dc81c28699b nodeName:}" failed. No retries permitted until 2026-07-17 21:17:34.83401955 +0000 UTC m=+35.603367528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls") pod "image-registry-69f9779877-887qk" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b") : secret "image-registry-tls" not found Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334067 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd7xv\" (UniqueName: \"kubernetes.io/projected/17c76d2b-6ab9-421d-97af-7c9e03181c7c-kube-api-access-wd7xv\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334103 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z89g\" (UniqueName: \"kubernetes.io/projected/133e55a6-f3de-428f-a9c6-fa697726d63a-kube-api-access-9z89g\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334126 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97cee0f9-881c-44b3-9c70-8dc81c28699b-ca-trust-extracted\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334155 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/999ecc4f-66b1-4ecb-97c5-c54040757d19-snapshots\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334179 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r598k\" (UniqueName: \"kubernetes.io/projected/fbe62ded-80ca-4471-ae23-dc1abdce5895-kube-api-access-r598k\") pod \"service-ca-operator-5bcbc94c78-qhb42\" (UID: \"fbe62ded-80ca-4471-ae23-dc1abdce5895\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" Jul 17 21:17:34.334211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334204 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999ecc4f-66b1-4ecb-97c5-c54040757d19-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334233 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fbe62ded-80ca-4471-ae23-dc1abdce5895-config\") pod \"service-ca-operator-5bcbc94c78-qhb42\" (UID: \"fbe62ded-80ca-4471-ae23-dc1abdce5895\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334258 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vh4r\" (UniqueName: \"kubernetes.io/projected/9cefc4af-8f20-4aad-ba30-041f244f9740-kube-api-access-4vh4r\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334287 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/036870c8-94b9-4906-8b4a-ea70695c9c79-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-w8cbj\" (UID: \"036870c8-94b9-4906-8b4a-ea70695c9c79\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334310 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-trusted-ca\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334345 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-default-certificate\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334371 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r9kd\" (UniqueName: \"kubernetes.io/projected/ec98f871-63b3-4790-bdcf-95a1409eb0c8-kube-api-access-5r9kd\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334398 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17c76d2b-6ab9-421d-97af-7c9e03181c7c-config-volume\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334429 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj5vv\" (UniqueName: \"kubernetes.io/projected/e0993ff9-f699-4050-b30e-1f92aedf96f2-kube-api-access-fj5vv\") pod \"network-check-source-7bc5b6bf85-jt7ll\" (UID: \"e0993ff9-f699-4050-b30e-1f92aedf96f2\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jt7ll" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334459 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmkw4\" (UniqueName: \"kubernetes.io/projected/389d5d02-d0be-4eff-bdea-bb0c695ca377-kube-api-access-nmkw4\") pod \"volume-data-source-validator-78cf4c6f6-ppljx\" (UID: \"389d5d02-d0be-4eff-bdea-bb0c695ca377\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ppljx" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334486 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-stats-auth\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334507 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a491e347-ad20-4da2-97bb-ce0d561846bb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334532 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/999ecc4f-66b1-4ecb-97c5-c54040757d19-tmp\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334539 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97cee0f9-881c-44b3-9c70-8dc81c28699b-ca-trust-extracted\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334580 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.334951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334604 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-certificates\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.335673 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334627 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999ecc4f-66b1-4ecb-97c5-c54040757d19-service-ca-bundle\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.335673 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334650 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cefc4af-8f20-4aad-ba30-041f244f9740-trusted-ca\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.335673 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334670 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cefc4af-8f20-4aad-ba30-041f244f9740-serving-cert\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.335673 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334696 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpkh7\" (UniqueName: \"kubernetes.io/projected/036870c8-94b9-4906-8b4a-ea70695c9c79-kube-api-access-kpkh7\") pod \"kube-storage-version-migrator-operator-7658d9b749-w8cbj\" (UID: \"036870c8-94b9-4906-8b4a-ea70695c9c79\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" Jul 17 21:17:34.335673 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334718 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mtct\" (UniqueName: \"kubernetes.io/projected/999ecc4f-66b1-4ecb-97c5-c54040757d19-kube-api-access-9mtct\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.335673 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.334940 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fbe62ded-80ca-4471-ae23-dc1abdce5895-config\") pod \"service-ca-operator-5bcbc94c78-qhb42\" (UID: \"fbe62ded-80ca-4471-ae23-dc1abdce5895\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" Jul 17 21:17:34.335673 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.335058 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/036870c8-94b9-4906-8b4a-ea70695c9c79-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-w8cbj\" (UID: \"036870c8-94b9-4906-8b4a-ea70695c9c79\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" Jul 17 21:17:34.335673 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.335314 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 21:17:34.335673 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.335377 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:17:34.835360537 +0000 UTC m=+35.604708519 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : secret "router-metrics-certs-default" not found Jul 17 21:17:34.335673 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.335502 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-certificates\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.336218 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.336181 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-trusted-ca\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.338637 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.338591 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fbe62ded-80ca-4471-ae23-dc1abdce5895-serving-cert\") pod \"service-ca-operator-5bcbc94c78-qhb42\" (UID: \"fbe62ded-80ca-4471-ae23-dc1abdce5895\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" Jul 17 21:17:34.338637 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.338597 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/036870c8-94b9-4906-8b4a-ea70695c9c79-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-w8cbj\" (UID: \"036870c8-94b9-4906-8b4a-ea70695c9c79\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" Jul 17 21:17:34.338806 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.338699 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-image-registry-private-configuration\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.338806 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.338796 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-installation-pull-secrets\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.338891 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.338861 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-stats-auth\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.339274 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.339255 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-default-certificate\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.343205 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.343183 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlcpd\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-kube-api-access-vlcpd\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.343564 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.343506 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-bound-sa-token\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.343876 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.343851 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z89g\" (UniqueName: \"kubernetes.io/projected/133e55a6-f3de-428f-a9c6-fa697726d63a-kube-api-access-9z89g\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.344526 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.344508 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpkh7\" (UniqueName: \"kubernetes.io/projected/036870c8-94b9-4906-8b4a-ea70695c9c79-kube-api-access-kpkh7\") pod \"kube-storage-version-migrator-operator-7658d9b749-w8cbj\" (UID: \"036870c8-94b9-4906-8b4a-ea70695c9c79\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" Jul 17 21:17:34.344711 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.344691 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmkw4\" (UniqueName: \"kubernetes.io/projected/389d5d02-d0be-4eff-bdea-bb0c695ca377-kube-api-access-nmkw4\") pod \"volume-data-source-validator-78cf4c6f6-ppljx\" (UID: \"389d5d02-d0be-4eff-bdea-bb0c695ca377\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ppljx" Jul 17 21:17:34.345220 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.345199 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r598k\" (UniqueName: \"kubernetes.io/projected/fbe62ded-80ca-4471-ae23-dc1abdce5895-kube-api-access-r598k\") pod \"service-ca-operator-5bcbc94c78-qhb42\" (UID: \"fbe62ded-80ca-4471-ae23-dc1abdce5895\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" Jul 17 21:17:34.376563 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.376499 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:34.420869 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.420836 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" Jul 17 21:17:34.431951 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.431922 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" Jul 17 21:17:34.435296 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435272 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.435415 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435307 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c76d2b-6ab9-421d-97af-7c9e03181c7c-tmp-dir\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.435415 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435353 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd7xv\" (UniqueName: \"kubernetes.io/projected/17c76d2b-6ab9-421d-97af-7c9e03181c7c-kube-api-access-wd7xv\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.435415 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435387 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/999ecc4f-66b1-4ecb-97c5-c54040757d19-snapshots\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.435582 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435415 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999ecc4f-66b1-4ecb-97c5-c54040757d19-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.435582 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.435420 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 21:17:34.435582 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435443 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vh4r\" (UniqueName: \"kubernetes.io/projected/9cefc4af-8f20-4aad-ba30-041f244f9740-kube-api-access-4vh4r\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.435582 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.435490 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls podName:17c76d2b-6ab9-421d-97af-7c9e03181c7c nodeName:}" failed. No retries permitted until 2026-07-17 21:17:34.935468061 +0000 UTC m=+35.704816023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls") pod "dns-default-rvqhc" (UID: "17c76d2b-6ab9-421d-97af-7c9e03181c7c") : secret "dns-default-metrics-tls" not found Jul 17 21:17:34.435582 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435512 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1f9a694-e7b7-4222-907d-583429aa5cf3-cert\") pod \"ingress-canary-hfktc\" (UID: \"e1f9a694-e7b7-4222-907d-583429aa5cf3\") " pod="openshift-ingress-canary/ingress-canary-hfktc" Jul 17 21:17:34.435582 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435564 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r9kd\" (UniqueName: \"kubernetes.io/projected/ec98f871-63b3-4790-bdcf-95a1409eb0c8-kube-api-access-5r9kd\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:17:34.435878 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435589 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17c76d2b-6ab9-421d-97af-7c9e03181c7c-config-volume\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.435878 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435617 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj5vv\" (UniqueName: \"kubernetes.io/projected/e0993ff9-f699-4050-b30e-1f92aedf96f2-kube-api-access-fj5vv\") pod \"network-check-source-7bc5b6bf85-jt7ll\" (UID: \"e0993ff9-f699-4050-b30e-1f92aedf96f2\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jt7ll" Jul 17 21:17:34.435878 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435665 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a491e347-ad20-4da2-97bb-ce0d561846bb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:34.435878 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435713 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/999ecc4f-66b1-4ecb-97c5-c54040757d19-tmp\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.435878 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435744 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c76d2b-6ab9-421d-97af-7c9e03181c7c-tmp-dir\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.435878 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435751 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999ecc4f-66b1-4ecb-97c5-c54040757d19-service-ca-bundle\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.435878 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435842 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cefc4af-8f20-4aad-ba30-041f244f9740-trusted-ca\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.435878 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435879 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cefc4af-8f20-4aad-ba30-041f244f9740-serving-cert\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.436253 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435914 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mtct\" (UniqueName: \"kubernetes.io/projected/999ecc4f-66b1-4ecb-97c5-c54040757d19-kube-api-access-9mtct\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.436253 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.435952 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cefc4af-8f20-4aad-ba30-041f244f9740-config\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.436253 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.436000 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/999ecc4f-66b1-4ecb-97c5-c54040757d19-serving-cert\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.436253 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.436033 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm84l\" (UniqueName: \"kubernetes.io/projected/a491e347-ad20-4da2-97bb-ce0d561846bb-kube-api-access-tm84l\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:34.436253 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.436064 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:17:34.436253 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.436098 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5sgf\" (UniqueName: \"kubernetes.io/projected/e1f9a694-e7b7-4222-907d-583429aa5cf3-kube-api-access-r5sgf\") pod \"ingress-canary-hfktc\" (UID: \"e1f9a694-e7b7-4222-907d-583429aa5cf3\") " pod="openshift-ingress-canary/ingress-canary-hfktc" Jul 17 21:17:34.436253 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.436206 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:34.436590 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.436320 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:34.436590 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.436367 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls podName:a491e347-ad20-4da2-97bb-ce0d561846bb nodeName:}" failed. No retries permitted until 2026-07-17 21:17:34.936351542 +0000 UTC m=+35.705699501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-94h2v" (UID: "a491e347-ad20-4da2-97bb-ce0d561846bb") : secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:34.436590 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.436361 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999ecc4f-66b1-4ecb-97c5-c54040757d19-service-ca-bundle\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.436590 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.436383 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/999ecc4f-66b1-4ecb-97c5-c54040757d19-snapshots\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.436799 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.436778 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 21:17:34.436851 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.436826 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls podName:ec98f871-63b3-4790-bdcf-95a1409eb0c8 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:34.93681173 +0000 UTC m=+35.706159692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gjwmm" (UID: "ec98f871-63b3-4790-bdcf-95a1409eb0c8") : secret "samples-operator-tls" not found Jul 17 21:17:34.437025 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.436999 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/999ecc4f-66b1-4ecb-97c5-c54040757d19-tmp\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.437362 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.437342 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cefc4af-8f20-4aad-ba30-041f244f9740-trusted-ca\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.437464 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.437441 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999ecc4f-66b1-4ecb-97c5-c54040757d19-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.437587 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.437570 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cefc4af-8f20-4aad-ba30-041f244f9740-config\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.439246 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.439223 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/999ecc4f-66b1-4ecb-97c5-c54040757d19-serving-cert\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.439702 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.439686 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cefc4af-8f20-4aad-ba30-041f244f9740-serving-cert\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.446440 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.446417 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a491e347-ad20-4da2-97bb-ce0d561846bb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:34.446787 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.446740 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17c76d2b-6ab9-421d-97af-7c9e03181c7c-config-volume\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.448607 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.448586 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd7xv\" (UniqueName: \"kubernetes.io/projected/17c76d2b-6ab9-421d-97af-7c9e03181c7c-kube-api-access-wd7xv\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.451343 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.451318 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ppljx" Jul 17 21:17:34.456253 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.456222 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm84l\" (UniqueName: \"kubernetes.io/projected/a491e347-ad20-4da2-97bb-ce0d561846bb-kube-api-access-tm84l\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:34.456600 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.456560 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mtct\" (UniqueName: \"kubernetes.io/projected/999ecc4f-66b1-4ecb-97c5-c54040757d19-kube-api-access-9mtct\") pod \"insights-operator-555d9d7885-k9wjf\" (UID: \"999ecc4f-66b1-4ecb-97c5-c54040757d19\") " pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.456669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.456651 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r9kd\" (UniqueName: \"kubernetes.io/projected/ec98f871-63b3-4790-bdcf-95a1409eb0c8-kube-api-access-5r9kd\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:17:34.456717 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.456676 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vh4r\" (UniqueName: \"kubernetes.io/projected/9cefc4af-8f20-4aad-ba30-041f244f9740-kube-api-access-4vh4r\") pod \"console-operator-565b7b569d-qdvms\" (UID: \"9cefc4af-8f20-4aad-ba30-041f244f9740\") " pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.457123 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.457094 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj5vv\" (UniqueName: \"kubernetes.io/projected/e0993ff9-f699-4050-b30e-1f92aedf96f2-kube-api-access-fj5vv\") pod \"network-check-source-7bc5b6bf85-jt7ll\" (UID: \"e0993ff9-f699-4050-b30e-1f92aedf96f2\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jt7ll" Jul 17 21:17:34.489587 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.489562 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-k9wjf" Jul 17 21:17:34.528084 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.527872 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:34.543829 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.540118 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1f9a694-e7b7-4222-907d-583429aa5cf3-cert\") pod \"ingress-canary-hfktc\" (UID: \"e1f9a694-e7b7-4222-907d-583429aa5cf3\") " pod="openshift-ingress-canary/ingress-canary-hfktc" Jul 17 21:17:34.543829 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.540243 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5sgf\" (UniqueName: \"kubernetes.io/projected/e1f9a694-e7b7-4222-907d-583429aa5cf3-kube-api-access-r5sgf\") pod \"ingress-canary-hfktc\" (UID: \"e1f9a694-e7b7-4222-907d-583429aa5cf3\") " pod="openshift-ingress-canary/ingress-canary-hfktc" Jul 17 21:17:34.544935 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.544884 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1f9a694-e7b7-4222-907d-583429aa5cf3-cert\") pod \"ingress-canary-hfktc\" (UID: \"e1f9a694-e7b7-4222-907d-583429aa5cf3\") " pod="openshift-ingress-canary/ingress-canary-hfktc" Jul 17 21:17:34.556696 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.556665 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5sgf\" (UniqueName: \"kubernetes.io/projected/e1f9a694-e7b7-4222-907d-583429aa5cf3-kube-api-access-r5sgf\") pod \"ingress-canary-hfktc\" (UID: \"e1f9a694-e7b7-4222-907d-583429aa5cf3\") " pod="openshift-ingress-canary/ingress-canary-hfktc" Jul 17 21:17:34.565141 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.560305 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jt7ll" Jul 17 21:17:34.584788 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.584761 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hfktc" Jul 17 21:17:34.672056 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.671970 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42"] Jul 17 21:17:34.694775 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.694703 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj"] Jul 17 21:17:34.700260 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:34.700220 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe62ded_80ca_4471_ae23_dc1abdce5895.slice/crio-bbc06c87932b9aad45e9b7f8b51b3147d4f4cf8880bff9aedcc226d389e64c84 WatchSource:0}: Error finding container bbc06c87932b9aad45e9b7f8b51b3147d4f4cf8880bff9aedcc226d389e64c84: Status 404 returned error can't find the container with id bbc06c87932b9aad45e9b7f8b51b3147d4f4cf8880bff9aedcc226d389e64c84 Jul 17 21:17:34.708762 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:34.708738 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod036870c8_94b9_4906_8b4a_ea70695c9c79.slice/crio-cc85cd8d42da4ea26bd4c3a75b0413275465d8f7b219c69609651cf777299856 WatchSource:0}: Error finding container cc85cd8d42da4ea26bd4c3a75b0413275465d8f7b219c69609651cf777299856: Status 404 returned error can't find the container with id cc85cd8d42da4ea26bd4c3a75b0413275465d8f7b219c69609651cf777299856 Jul 17 21:17:34.715876 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.713918 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:17:34.715876 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.714188 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:17:34.715876 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.714317 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:17:34.717238 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.717077 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lpmh2\"" Jul 17 21:17:34.717787 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.717334 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 21:17:34.717787 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.717533 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 21:17:34.717787 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.717533 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jrzlv\"" Jul 17 21:17:34.732307 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.731257 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ppljx"] Jul 17 21:17:34.739495 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:34.737717 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod389d5d02_d0be_4eff_bdea_bb0c695ca377.slice/crio-1c33e7019c0662e05fe7e23a4c047ac9c600f668c531f364b7558fbff7216d3b WatchSource:0}: Error finding container 1c33e7019c0662e05fe7e23a4c047ac9c600f668c531f364b7558fbff7216d3b: Status 404 returned error can't find the container with id 1c33e7019c0662e05fe7e23a4c047ac9c600f668c531f364b7558fbff7216d3b Jul 17 21:17:34.752518 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.752494 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-k9wjf"] Jul 17 21:17:34.759660 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:34.759531 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod999ecc4f_66b1_4ecb_97c5_c54040757d19.slice/crio-829a48987fd9addf6c586cddb25ac1662e7abf3d61d2caa870d794d763ad80cb WatchSource:0}: Error finding container 829a48987fd9addf6c586cddb25ac1662e7abf3d61d2caa870d794d763ad80cb: Status 404 returned error can't find the container with id 829a48987fd9addf6c586cddb25ac1662e7abf3d61d2caa870d794d763ad80cb Jul 17 21:17:34.783278 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.783252 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-qdvms"] Jul 17 21:17:34.786884 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:34.786848 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cefc4af_8f20_4aad_ba30_041f244f9740.slice/crio-cd524fecc816a0d22f229e67c9bfd9a13677e0a1ed76f5c792861fffc3b6642d WatchSource:0}: Error finding container cd524fecc816a0d22f229e67c9bfd9a13677e0a1ed76f5c792861fffc3b6642d: Status 404 returned error can't find the container with id cd524fecc816a0d22f229e67c9bfd9a13677e0a1ed76f5c792861fffc3b6642d Jul 17 21:17:34.796474 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.796450 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-jt7ll"] Jul 17 21:17:34.802369 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:34.802344 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0993ff9_f699_4050_b30e_1f92aedf96f2.slice/crio-a7289ea6f867597e618ceab6cb60ceb498064c8c49ced85dbdf08364bd32f67d WatchSource:0}: Error finding container a7289ea6f867597e618ceab6cb60ceb498064c8c49ced85dbdf08364bd32f67d: Status 404 returned error can't find the container with id a7289ea6f867597e618ceab6cb60ceb498064c8c49ced85dbdf08364bd32f67d Jul 17 21:17:34.820174 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.820152 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hfktc"] Jul 17 21:17:34.823178 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:17:34.823153 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f9a694_e7b7_4222_907d_583429aa5cf3.slice/crio-d2d12f6ae4dd0706189f25937ef24ef3c3f0c202e98d34baac63602c17802da1 WatchSource:0}: Error finding container d2d12f6ae4dd0706189f25937ef24ef3c3f0c202e98d34baac63602c17802da1: Status 404 returned error can't find the container with id d2d12f6ae4dd0706189f25937ef24ef3c3f0c202e98d34baac63602c17802da1 Jul 17 21:17:34.846232 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.846207 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.846334 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.846286 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:34.846393 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.846369 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:17:35.846346177 +0000 UTC m=+36.615694153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : configmap references non-existent config key: service-ca.crt Jul 17 21:17:34.846457 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.846423 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 21:17:34.846457 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.846435 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f9779877-887qk: secret "image-registry-tls" not found Jul 17 21:17:34.846578 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.846479 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls podName:97cee0f9-881c-44b3-9c70-8dc81c28699b nodeName:}" failed. No retries permitted until 2026-07-17 21:17:35.846464955 +0000 UTC m=+36.615812913 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls") pod "image-registry-69f9779877-887qk" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b") : secret "image-registry-tls" not found Jul 17 21:17:34.846578 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.846502 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:34.846680 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.846614 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 21:17:34.846680 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.846651 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:17:35.846639221 +0000 UTC m=+36.615987179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : secret "router-metrics-certs-default" not found Jul 17 21:17:34.912334 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.912291 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-k9wjf" event={"ID":"999ecc4f-66b1-4ecb-97c5-c54040757d19","Type":"ContainerStarted","Data":"829a48987fd9addf6c586cddb25ac1662e7abf3d61d2caa870d794d763ad80cb"} Jul 17 21:17:34.913506 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.913479 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" event={"ID":"9cefc4af-8f20-4aad-ba30-041f244f9740","Type":"ContainerStarted","Data":"cd524fecc816a0d22f229e67c9bfd9a13677e0a1ed76f5c792861fffc3b6642d"} Jul 17 21:17:34.914672 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.914646 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" event={"ID":"036870c8-94b9-4906-8b4a-ea70695c9c79","Type":"ContainerStarted","Data":"cc85cd8d42da4ea26bd4c3a75b0413275465d8f7b219c69609651cf777299856"} Jul 17 21:17:34.915867 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.915839 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" event={"ID":"fbe62ded-80ca-4471-ae23-dc1abdce5895","Type":"ContainerStarted","Data":"bbc06c87932b9aad45e9b7f8b51b3147d4f4cf8880bff9aedcc226d389e64c84"} Jul 17 21:17:34.917198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.917177 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hfktc" event={"ID":"e1f9a694-e7b7-4222-907d-583429aa5cf3","Type":"ContainerStarted","Data":"d2d12f6ae4dd0706189f25937ef24ef3c3f0c202e98d34baac63602c17802da1"} Jul 17 21:17:34.918460 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.918429 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jt7ll" event={"ID":"e0993ff9-f699-4050-b30e-1f92aedf96f2","Type":"ContainerStarted","Data":"a7289ea6f867597e618ceab6cb60ceb498064c8c49ced85dbdf08364bd32f67d"} Jul 17 21:17:34.919607 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.919583 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ppljx" event={"ID":"389d5d02-d0be-4eff-bdea-bb0c695ca377","Type":"ContainerStarted","Data":"1c33e7019c0662e05fe7e23a4c047ac9c600f668c531f364b7558fbff7216d3b"} Jul 17 21:17:34.947154 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.947131 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:17:34.947281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.947192 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:34.947281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:34.947222 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:34.947394 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.947299 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 21:17:34.947394 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.947343 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 21:17:34.947394 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.947357 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:34.947394 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.947376 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls podName:ec98f871-63b3-4790-bdcf-95a1409eb0c8 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:35.94735599 +0000 UTC m=+36.716703961 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gjwmm" (UID: "ec98f871-63b3-4790-bdcf-95a1409eb0c8") : secret "samples-operator-tls" not found Jul 17 21:17:34.947394 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.947394 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls podName:17c76d2b-6ab9-421d-97af-7c9e03181c7c nodeName:}" failed. No retries permitted until 2026-07-17 21:17:35.947385367 +0000 UTC m=+36.716733326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls") pod "dns-default-rvqhc" (UID: "17c76d2b-6ab9-421d-97af-7c9e03181c7c") : secret "dns-default-metrics-tls" not found Jul 17 21:17:34.947630 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:34.947425 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls podName:a491e347-ad20-4da2-97bb-ce0d561846bb nodeName:}" failed. No retries permitted until 2026-07-17 21:17:35.947408409 +0000 UTC m=+36.716756374 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-94h2v" (UID: "a491e347-ad20-4da2-97bb-ce0d561846bb") : secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:35.857136 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:35.856508 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:35.857136 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:35.856610 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:35.857136 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:35.856670 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:35.857136 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.856865 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 21:17:35.857136 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.856886 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f9779877-887qk: secret "image-registry-tls" not found Jul 17 21:17:35.857136 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.856942 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls podName:97cee0f9-881c-44b3-9c70-8dc81c28699b nodeName:}" failed. No retries permitted until 2026-07-17 21:17:37.856924781 +0000 UTC m=+38.626272754 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls") pod "image-registry-69f9779877-887qk" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b") : secret "image-registry-tls" not found Jul 17 21:17:35.857136 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.857005 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 21:17:35.857136 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.857038 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:17:37.85702731 +0000 UTC m=+38.626375267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : secret "router-metrics-certs-default" not found Jul 17 21:17:35.857136 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.857111 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:17:37.857101858 +0000 UTC m=+38.626449819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : configmap references non-existent config key: service-ca.crt Jul 17 21:17:35.958686 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:35.957707 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:35.958686 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:35.957861 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:17:35.958686 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:35.957917 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:35.958686 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.958039 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:35.958686 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.958100 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls podName:a491e347-ad20-4da2-97bb-ce0d561846bb nodeName:}" failed. No retries permitted until 2026-07-17 21:17:37.958081913 +0000 UTC m=+38.727429872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-94h2v" (UID: "a491e347-ad20-4da2-97bb-ce0d561846bb") : secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:35.958686 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.958501 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 21:17:35.958686 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.958568 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls podName:17c76d2b-6ab9-421d-97af-7c9e03181c7c nodeName:}" failed. No retries permitted until 2026-07-17 21:17:37.958535453 +0000 UTC m=+38.727883411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls") pod "dns-default-rvqhc" (UID: "17c76d2b-6ab9-421d-97af-7c9e03181c7c") : secret "dns-default-metrics-tls" not found Jul 17 21:17:35.958686 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.958624 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 21:17:35.958686 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:35.958661 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls podName:ec98f871-63b3-4790-bdcf-95a1409eb0c8 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:37.95864887 +0000 UTC m=+38.727996829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gjwmm" (UID: "ec98f871-63b3-4790-bdcf-95a1409eb0c8") : secret "samples-operator-tls" not found Jul 17 21:17:36.993688 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:36.993592 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:37.063742 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:37.063706 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:37.875188 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:37.875156 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:37.875357 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:37.875259 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:37.875357 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:37.875297 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:37.875440 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.875416 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:17:41.875403323 +0000 UTC m=+42.644751279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : configmap references non-existent config key: service-ca.crt Jul 17 21:17:37.875650 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.875627 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 21:17:37.875730 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.875685 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:17:41.875669695 +0000 UTC m=+42.645017670 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : secret "router-metrics-certs-default" not found Jul 17 21:17:37.875788 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.875751 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 21:17:37.875788 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.875760 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f9779877-887qk: secret "image-registry-tls" not found Jul 17 21:17:37.875788 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.875786 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls podName:97cee0f9-881c-44b3-9c70-8dc81c28699b nodeName:}" failed. No retries permitted until 2026-07-17 21:17:41.875777913 +0000 UTC m=+42.645125868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls") pod "image-registry-69f9779877-887qk" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b") : secret "image-registry-tls" not found Jul 17 21:17:37.976625 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:37.975740 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:37.976625 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:37.975790 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:37.976625 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:37.975929 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:17:37.976625 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.976060 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 21:17:37.976625 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.976116 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls podName:ec98f871-63b3-4790-bdcf-95a1409eb0c8 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:41.97609924 +0000 UTC m=+42.745447199 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gjwmm" (UID: "ec98f871-63b3-4790-bdcf-95a1409eb0c8") : secret "samples-operator-tls" not found Jul 17 21:17:37.976625 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.976184 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:37.976625 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.976220 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls podName:a491e347-ad20-4da2-97bb-ce0d561846bb nodeName:}" failed. No retries permitted until 2026-07-17 21:17:41.976207778 +0000 UTC m=+42.745555739 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-94h2v" (UID: "a491e347-ad20-4da2-97bb-ce0d561846bb") : secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:37.976625 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.976276 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 21:17:37.976625 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:37.976305 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls podName:17c76d2b-6ab9-421d-97af-7c9e03181c7c nodeName:}" failed. No retries permitted until 2026-07-17 21:17:41.976295363 +0000 UTC m=+42.745643322 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls") pod "dns-default-rvqhc" (UID: "17c76d2b-6ab9-421d-97af-7c9e03181c7c") : secret "dns-default-metrics-tls" not found Jul 17 21:17:38.016429 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.016395 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr"] Jul 17 21:17:38.018738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.018717 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:17:38.021635 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.021608 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 17 21:17:38.021723 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.021634 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 17 21:17:38.022836 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.022688 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8c2tv\"" Jul 17 21:17:38.028661 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.028126 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr"] Jul 17 21:17:38.035158 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.035087 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:38.132866 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.132758 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:38.177995 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.177908 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:17:38.178152 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.178123 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:17:38.233972 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.233923 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:38.279358 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.278863 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:17:38.279358 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.278974 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:17:38.279358 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:38.279096 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 21:17:38.279358 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:38.279151 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert podName:dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:38.779136447 +0000 UTC m=+39.548484403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-vt6tr" (UID: "dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71") : secret "networking-console-plugin-cert" not found Jul 17 21:17:38.279718 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.279411 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:17:38.344422 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.344380 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:38.407136 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.407014 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:38.784321 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.784269 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:17:38.784499 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:38.784430 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 21:17:38.784576 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:38.784518 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert podName:dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:39.784496559 +0000 UTC m=+40.553844522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-vt6tr" (UID: "dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71") : secret "networking-console-plugin-cert" not found Jul 17 21:17:38.934583 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:38.934532 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:39.052659 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:39.052580 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:39.152692 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:39.152664 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:39.254462 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:39.254421 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:39.794644 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:39.794599 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:17:39.794829 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:39.794779 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 21:17:39.794881 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:39.794867 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert podName:dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:41.794849548 +0000 UTC m=+42.564197508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-vt6tr" (UID: "dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71") : secret "networking-console-plugin-cert" not found Jul 17 21:17:39.954538 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:39.954500 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:40.863351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:40.863294 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:40.962842 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:40.962819 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:41.815139 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:41.815099 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:17:41.815303 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:41.815247 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 21:17:41.815358 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:41.815321 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert podName:dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:45.815301975 +0000 UTC m=+46.584649947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-vt6tr" (UID: "dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71") : secret "networking-console-plugin-cert" not found Jul 17 21:17:41.915832 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:41.915798 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:41.916276 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:41.915858 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:41.916276 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:41.915919 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:41.916276 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:41.915961 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 21:17:41.916276 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:41.915982 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:17:49.915962631 +0000 UTC m=+50.685310597 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : configmap references non-existent config key: service-ca.crt Jul 17 21:17:41.916276 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:41.916008 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:17:49.915997177 +0000 UTC m=+50.685345153 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : secret "router-metrics-certs-default" not found Jul 17 21:17:41.916276 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:41.916052 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 21:17:41.916276 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:41.916066 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f9779877-887qk: secret "image-registry-tls" not found Jul 17 21:17:41.916276 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:41.916116 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls podName:97cee0f9-881c-44b3-9c70-8dc81c28699b nodeName:}" failed. No retries permitted until 2026-07-17 21:17:49.916098719 +0000 UTC m=+50.685446692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls") pod "image-registry-69f9779877-887qk" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b") : secret "image-registry-tls" not found Jul 17 21:17:42.017230 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:42.017197 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:17:42.017429 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:42.017262 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:42.017429 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:42.017303 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:42.017529 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:42.017484 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 21:17:42.017604 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:42.017567 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls podName:17c76d2b-6ab9-421d-97af-7c9e03181c7c nodeName:}" failed. No retries permitted until 2026-07-17 21:17:50.017528575 +0000 UTC m=+50.786876534 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls") pod "dns-default-rvqhc" (UID: "17c76d2b-6ab9-421d-97af-7c9e03181c7c") : secret "dns-default-metrics-tls" not found Jul 17 21:17:42.017840 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:42.017764 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:42.017840 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:42.017818 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls podName:a491e347-ad20-4da2-97bb-ce0d561846bb nodeName:}" failed. No retries permitted until 2026-07-17 21:17:50.017804799 +0000 UTC m=+50.787152760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-94h2v" (UID: "a491e347-ad20-4da2-97bb-ce0d561846bb") : secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:42.017840 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:42.017816 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 21:17:42.018034 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:42.017887 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls podName:ec98f871-63b3-4790-bdcf-95a1409eb0c8 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:50.017869469 +0000 UTC m=+50.787217427 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gjwmm" (UID: "ec98f871-63b3-4790-bdcf-95a1409eb0c8") : secret "samples-operator-tls" not found Jul 17 21:17:42.626095 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:42.626055 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:42.729897 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:42.729861 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:43.264073 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.264047 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:43.323872 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.323807 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:43.949674 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.949645 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/0.log" Jul 17 21:17:43.949863 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.949694 2554 generic.go:356] "Generic (PLEG): container finished" podID="9cefc4af-8f20-4aad-ba30-041f244f9740" containerID="d00b5337fabd90a04e0b9dd2b9309687ac28585207d088172efd0db41f3b09f4" exitCode=255 Jul 17 21:17:43.949863 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.949785 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" event={"ID":"9cefc4af-8f20-4aad-ba30-041f244f9740","Type":"ContainerDied","Data":"d00b5337fabd90a04e0b9dd2b9309687ac28585207d088172efd0db41f3b09f4"} Jul 17 21:17:43.950018 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.950001 2554 scope.go:117] "RemoveContainer" containerID="d00b5337fabd90a04e0b9dd2b9309687ac28585207d088172efd0db41f3b09f4" Jul 17 21:17:43.951720 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.951689 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" event={"ID":"036870c8-94b9-4906-8b4a-ea70695c9c79","Type":"ContainerStarted","Data":"3bb22d5616f92235e56b077cae62acc8bea120e6fb984e8bb52e3993d8f82036"} Jul 17 21:17:43.953611 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.953588 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" event={"ID":"fbe62ded-80ca-4471-ae23-dc1abdce5895","Type":"ContainerStarted","Data":"f3677263abcb87e9716f741c221449aeabeff5e5f6dcdbeb7cac982c6e790e94"} Jul 17 21:17:43.957462 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.956985 2554 generic.go:356] "Generic (PLEG): container finished" podID="6b870da3-94e0-4301-8be9-17174f8a012a" containerID="e653635d9c67a53b18bbfca6b5c51dcb92df120ff9e1a8c04279c2bcda239aae" exitCode=0 Jul 17 21:17:43.957462 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.957179 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkkg4" event={"ID":"6b870da3-94e0-4301-8be9-17174f8a012a","Type":"ContainerDied","Data":"e653635d9c67a53b18bbfca6b5c51dcb92df120ff9e1a8c04279c2bcda239aae"} Jul 17 21:17:43.964855 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.960070 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hfktc" event={"ID":"e1f9a694-e7b7-4222-907d-583429aa5cf3","Type":"ContainerStarted","Data":"5d1a0f316c905acfc2030b73c0a9ce905598a8d558145be904843b24aa6d78fa"} Jul 17 21:17:43.964855 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.961625 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jt7ll" event={"ID":"e0993ff9-f699-4050-b30e-1f92aedf96f2","Type":"ContainerStarted","Data":"1f4cc492bb4831e9113f77999fba7ba559b7ea8de2a2d53672020fc9ace77f7f"} Jul 17 21:17:43.965828 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.965425 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ppljx" event={"ID":"389d5d02-d0be-4eff-bdea-bb0c695ca377","Type":"ContainerStarted","Data":"70e3f3fd98d7c264d1600da34efc11df42a633706e697d8f9e29428c3c499a02"} Jul 17 21:17:43.969044 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.968494 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-k9wjf" event={"ID":"999ecc4f-66b1-4ecb-97c5-c54040757d19","Type":"ContainerStarted","Data":"a0ba0873ca6185efad43c7e620dcdf567ba20945cc4f3cf59b9d9dee03c98b36"} Jul 17 21:17:43.985442 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:43.985359 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" podStartSLOduration=6.286739506 podStartE2EDuration="14.985341369s" totalImagesPullingTime="8.698601863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:43.985261572 +0000 UTC m=+44.754609551" watchObservedRunningTime="2026-07-17 21:17:43.985341369 +0000 UTC m=+44.754689329" Jul 17 21:17:44.029308 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.028366 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" podStartSLOduration=6.334528557 podStartE2EDuration="15.028343799s" totalImagesPullingTime="8.693815242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:44.003588012 +0000 UTC m=+44.772935981" watchObservedRunningTime="2026-07-17 21:17:44.028343799 +0000 UTC m=+44.797691779" Jul 17 21:17:44.075872 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.074453 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hfktc" podStartSLOduration=1.491895295 podStartE2EDuration="10.074431842s" totalImagesPullingTime="8.582536547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:44.044769155 +0000 UTC m=+44.814117135" watchObservedRunningTime="2026-07-17 21:17:44.074431842 +0000 UTC m=+44.843779821" Jul 17 21:17:44.075872 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.075019 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ppljx" podStartSLOduration=6.407725844 podStartE2EDuration="15.075003054s" totalImagesPullingTime="8.66727721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:44.073997105 +0000 UTC m=+44.843345084" watchObservedRunningTime="2026-07-17 21:17:44.075003054 +0000 UTC m=+44.844351033" Jul 17 21:17:44.118878 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.118813 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jt7ll" podStartSLOduration=6.515603327 podStartE2EDuration="15.118765322s" totalImagesPullingTime="8.603161995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:44.09486676 +0000 UTC m=+44.864214739" watchObservedRunningTime="2026-07-17 21:17:44.118765322 +0000 UTC m=+44.888113301" Jul 17 21:17:44.188891 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.188848 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:44.290072 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.290038 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:44.362748 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.362676 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:44.528258 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.528224 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:44.528428 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.528316 2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:44.859875 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.859804 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-k9wjf" podStartSLOduration=7.207319862 podStartE2EDuration="15.859778171s" totalImagesPullingTime="8.652458309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:44.11861483 +0000 UTC m=+44.887962809" watchObservedRunningTime="2026-07-17 21:17:44.859778171 +0000 UTC m=+45.629126150" Jul 17 21:17:44.860085 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.860065 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jxrcv"] Jul 17 21:17:44.862156 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.862129 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:44.866642 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.866358 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 17 21:17:44.866642 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.866434 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rvj24\"" Jul 17 21:17:44.866642 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.866360 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 17 21:17:44.880523 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.880493 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jxrcv"] Jul 17 21:17:44.929125 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.929095 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:44.951220 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.951196 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-data-volume\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:44.951372 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.951235 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:44.951372 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.951287 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-crio-socket\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:44.951372 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.951314 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbjrn\" (UniqueName: \"kubernetes.io/projected/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-api-access-rbjrn\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:44.951531 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.951456 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:44.974395 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.974368 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/1.log" Jul 17 21:17:44.974795 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.974778 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/0.log" Jul 17 21:17:44.974894 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.974817 2554 generic.go:356] "Generic (PLEG): container finished" podID="9cefc4af-8f20-4aad-ba30-041f244f9740" containerID="fa3d8bf483210505d4f08727cda36695e27a97272126dac26aa742b2997c7c08" exitCode=255 Jul 17 21:17:44.974894 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.974873 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" event={"ID":"9cefc4af-8f20-4aad-ba30-041f244f9740","Type":"ContainerDied","Data":"fa3d8bf483210505d4f08727cda36695e27a97272126dac26aa742b2997c7c08"} Jul 17 21:17:44.974996 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.974946 2554 scope.go:117] "RemoveContainer" containerID="d00b5337fabd90a04e0b9dd2b9309687ac28585207d088172efd0db41f3b09f4" Jul 17 21:17:44.975146 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.975122 2554 scope.go:117] "RemoveContainer" containerID="fa3d8bf483210505d4f08727cda36695e27a97272126dac26aa742b2997c7c08" Jul 17 21:17:44.975344 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:44.975320 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-qdvms_openshift-console-operator(9cefc4af-8f20-4aad-ba30-041f244f9740)\"" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" podUID="9cefc4af-8f20-4aad-ba30-041f244f9740" Jul 17 21:17:44.978288 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.978268 2554 generic.go:356] "Generic (PLEG): container finished" podID="6b870da3-94e0-4301-8be9-17174f8a012a" containerID="c40733c749f52358e13489188eaf81d8a0f55125c4110be649f4345b1e118d6a" exitCode=0 Jul 17 21:17:44.978409 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:44.978373 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkkg4" event={"ID":"6b870da3-94e0-4301-8be9-17174f8a012a","Type":"ContainerDied","Data":"c40733c749f52358e13489188eaf81d8a0f55125c4110be649f4345b1e118d6a"} Jul 17 21:17:45.005493 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.005464 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:45.053592 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.052387 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-crio-socket\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:45.053592 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.052439 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbjrn\" (UniqueName: \"kubernetes.io/projected/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-api-access-rbjrn\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:45.053592 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.052649 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:45.053592 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.052773 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-data-volume\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:45.053592 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:45.053134 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 21:17:45.053592 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.053149 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:45.053592 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:45.053189 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm podName:fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f nodeName:}" failed. No retries permitted until 2026-07-17 21:17:45.553174316 +0000 UTC m=+46.322522271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxrcv" (UID: "fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f") : configmap "kube-rbac-proxy" not found Jul 17 21:17:45.053592 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:45.053241 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 21:17:45.053592 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:45.053431 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls podName:fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f nodeName:}" failed. No retries permitted until 2026-07-17 21:17:45.553421886 +0000 UTC m=+46.322769842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxrcv" (UID: "fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f") : secret "insights-runtime-extractor-tls" not found Jul 17 21:17:45.053592 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.053581 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-crio-socket\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:45.054629 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.054579 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-data-volume\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:45.073972 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.073949 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbjrn\" (UniqueName: \"kubernetes.io/projected/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-api-access-rbjrn\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:45.155874 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.155784 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:45.266374 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.266336 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:45.328338 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.328309 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:45.559498 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.559470 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:45.559697 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.559540 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:45.559697 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:45.559651 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 21:17:45.559697 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:45.559660 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 21:17:45.559844 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:45.559718 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls podName:fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f nodeName:}" failed. No retries permitted until 2026-07-17 21:17:46.559705937 +0000 UTC m=+47.329053893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxrcv" (UID: "fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f") : secret "insights-runtime-extractor-tls" not found Jul 17 21:17:45.559844 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:45.559733 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm podName:fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f nodeName:}" failed. No retries permitted until 2026-07-17 21:17:46.559727661 +0000 UTC m=+47.329075617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxrcv" (UID: "fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f") : configmap "kube-rbac-proxy" not found Jul 17 21:17:45.645076 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.645046 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:45.727606 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.727572 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:45.862363 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.862271 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:17:45.862524 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:45.862422 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 21:17:45.862524 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:45.862499 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert podName:dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71 nodeName:}" failed. No retries permitted until 2026-07-17 21:17:53.86247922 +0000 UTC m=+54.631827177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-vt6tr" (UID: "dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71") : secret "networking-console-plugin-cert" not found Jul 17 21:17:45.925835 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.925808 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:45.985757 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.985727 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkkg4" event={"ID":"6b870da3-94e0-4301-8be9-17174f8a012a","Type":"ContainerStarted","Data":"2ca4d2ef670a530f9f55b26755d82df9a2ae3644eb0daa731d8f0a1ccdb220f7"} Jul 17 21:17:45.987172 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.987152 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/1.log" Jul 17 21:17:45.987423 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:45.987409 2554 scope.go:117] "RemoveContainer" containerID="fa3d8bf483210505d4f08727cda36695e27a97272126dac26aa742b2997c7c08" Jul 17 21:17:45.987586 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:45.987569 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-qdvms_openshift-console-operator(9cefc4af-8f20-4aad-ba30-041f244f9740)\"" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" podUID="9cefc4af-8f20-4aad-ba30-041f244f9740" Jul 17 21:17:46.013844 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:46.013802 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fkkg4" podStartSLOduration=9.116004859 podStartE2EDuration="46.013790212s" totalImagesPullingTime="36.897785353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:17:46.011838653 +0000 UTC m=+46.781186629" watchObservedRunningTime="2026-07-17 21:17:46.013790212 +0000 UTC m=+46.783138190" Jul 17 21:17:46.019529 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:46.019499 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:46.124848 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:46.124769 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:46.195409 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:46.195384 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:46.569356 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:46.569322 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:46.569762 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:46.569428 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:46.569762 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:46.569488 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 21:17:46.569762 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:46.569534 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 21:17:46.569762 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:46.569582 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls podName:fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f nodeName:}" failed. No retries permitted until 2026-07-17 21:17:48.569566461 +0000 UTC m=+49.338914417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxrcv" (UID: "fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f") : secret "insights-runtime-extractor-tls" not found Jul 17 21:17:46.569762 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:46.569600 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm podName:fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f nodeName:}" failed. No retries permitted until 2026-07-17 21:17:48.569588277 +0000 UTC m=+49.338936233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxrcv" (UID: "fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f") : configmap "kube-rbac-proxy" not found Jul 17 21:17:46.659906 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:46.659880 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rjxhs_768859c6-079b-4947-ad85-d6bf78ac1023/dns-node-resolver/0.log" Jul 17 21:17:47.259090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:47.259061 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6zztr_82b932c6-788d-4a01-bdcb-c8c9b7303984/node-ca/0.log" Jul 17 21:17:47.663984 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:47.663950 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:47.781046 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:47.781012 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:47.912966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:47.912934 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:48.068512 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:48.068476 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hfktc_e1f9a694-e7b7-4222-907d-583429aa5cf3/serve-healthcheck-canary/0.log" Jul 17 21:17:48.587560 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:48.587522 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:48.587727 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:48.587647 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:48.587727 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:48.587680 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 21:17:48.587727 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:48.587710 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 21:17:48.587823 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:48.587759 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls podName:fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f nodeName:}" failed. No retries permitted until 2026-07-17 21:17:52.587741847 +0000 UTC m=+53.357089824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxrcv" (UID: "fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f") : secret "insights-runtime-extractor-tls" not found Jul 17 21:17:48.587823 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:48.587777 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm podName:fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f nodeName:}" failed. No retries permitted until 2026-07-17 21:17:52.587768995 +0000 UTC m=+53.357116951 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxrcv" (UID: "fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f") : configmap "kube-rbac-proxy" not found Jul 17 21:17:48.932904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:48.932829 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:49.998145 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:49.998104 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:49.998513 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:49.998165 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:17:49.998513 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:49.998271 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:18:05.998253111 +0000 UTC m=+66.767601068 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : configmap references non-existent config key: service-ca.crt Jul 17 21:17:49.998513 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:49.998300 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:17:49.998513 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:49.998320 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 21:17:49.998513 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:49.998336 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f9779877-887qk: secret "image-registry-tls" not found Jul 17 21:17:49.998513 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:49.998375 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls podName:97cee0f9-881c-44b3-9c70-8dc81c28699b nodeName:}" failed. No retries permitted until 2026-07-17 21:18:05.998363267 +0000 UTC m=+66.767711229 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls") pod "image-registry-69f9779877-887qk" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b") : secret "image-registry-tls" not found Jul 17 21:17:49.998513 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:49.998395 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 17 21:17:49.998513 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:49.998438 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs podName:133e55a6-f3de-428f-a9c6-fa697726d63a nodeName:}" failed. No retries permitted until 2026-07-17 21:18:05.998430064 +0000 UTC m=+66.767778020 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs") pod "router-default-74bdb64d9b-rxs6z" (UID: "133e55a6-f3de-428f-a9c6-fa697726d63a") : secret "router-metrics-certs-default" not found Jul 17 21:17:50.099075 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:50.099032 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:17:50.099236 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:50.099091 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:17:50.099236 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:50.099120 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:17:50.099236 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:50.099176 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 21:17:50.099236 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:50.099234 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 21:17:50.099357 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:50.099241 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls podName:ec98f871-63b3-4790-bdcf-95a1409eb0c8 nodeName:}" failed. No retries permitted until 2026-07-17 21:18:06.099226612 +0000 UTC m=+66.868574572 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gjwmm" (UID: "ec98f871-63b3-4790-bdcf-95a1409eb0c8") : secret "samples-operator-tls" not found Jul 17 21:17:50.099357 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:50.099236 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:50.099357 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:50.099280 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls podName:17c76d2b-6ab9-421d-97af-7c9e03181c7c nodeName:}" failed. No retries permitted until 2026-07-17 21:18:06.099269741 +0000 UTC m=+66.868617697 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls") pod "dns-default-rvqhc" (UID: "17c76d2b-6ab9-421d-97af-7c9e03181c7c") : secret "dns-default-metrics-tls" not found Jul 17 21:17:50.099357 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:50.099299 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls podName:a491e347-ad20-4da2-97bb-ce0d561846bb nodeName:}" failed. No retries permitted until 2026-07-17 21:18:06.099287644 +0000 UTC m=+66.868635600 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-94h2v" (UID: "a491e347-ad20-4da2-97bb-ce0d561846bb") : secret "cluster-monitoring-operator-tls" not found Jul 17 21:17:50.532713 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:50.532669 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:52.051650 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:52.051612 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:52.252158 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:52.252126 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:52.369494 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:52.369417 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:52.472075 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:52.472040 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:52.577327 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:52.577303 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:52.621242 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:52.621165 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:52.621375 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:52.621283 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:52.621375 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:52.621319 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 21:17:52.621439 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:52.621376 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls podName:fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f nodeName:}" failed. No retries permitted until 2026-07-17 21:18:00.621362645 +0000 UTC m=+61.390710604 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxrcv" (UID: "fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f") : secret "insights-runtime-extractor-tls" not found Jul 17 21:17:52.621760 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:52.621743 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:17:52.695274 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:52.695238 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:52.802592 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:52.802562 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:52.927866 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:52.927787 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:53.932390 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:53.932360 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:17:53.932812 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:53.932508 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 17 21:17:53.932812 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:53.932588 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert podName:dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71 nodeName:}" failed. No retries permitted until 2026-07-17 21:18:09.932574248 +0000 UTC m=+70.701922203 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-vt6tr" (UID: "dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71") : secret "networking-console-plugin-cert" not found Jul 17 21:17:53.943705 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:53.943680 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:54.223984 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:54.223904 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:54.353134 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:54.353099 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:54.512410 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:54.512327 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:54.528253 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:54.528231 2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:54.528366 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:54.528314 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:17:54.528695 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:54.528682 2554 scope.go:117] "RemoveContainer" containerID="fa3d8bf483210505d4f08727cda36695e27a97272126dac26aa742b2997c7c08" Jul 17 21:17:54.746777 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:54.746749 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:54.905125 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:54.905091 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:54.969336 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:54.969303 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:55.017194 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:55.017172 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:17:55.017637 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:55.017615 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/1.log" Jul 17 21:17:55.017700 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:55.017654 2554 generic.go:356] "Generic (PLEG): container finished" podID="9cefc4af-8f20-4aad-ba30-041f244f9740" containerID="1f6ae1df854d0b245811494d7524b61b486b686f8d04e4fa11ef02e855605b82" exitCode=255 Jul 17 21:17:55.017735 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:55.017702 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" event={"ID":"9cefc4af-8f20-4aad-ba30-041f244f9740","Type":"ContainerDied","Data":"1f6ae1df854d0b245811494d7524b61b486b686f8d04e4fa11ef02e855605b82"} Jul 17 21:17:55.017764 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:55.017753 2554 scope.go:117] "RemoveContainer" containerID="fa3d8bf483210505d4f08727cda36695e27a97272126dac26aa742b2997c7c08" Jul 17 21:17:55.018100 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:55.018084 2554 scope.go:117] "RemoveContainer" containerID="1f6ae1df854d0b245811494d7524b61b486b686f8d04e4fa11ef02e855605b82" Jul 17 21:17:55.018250 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:17:55.018229 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-qdvms_openshift-console-operator(9cefc4af-8f20-4aad-ba30-041f244f9740)\"" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" podUID="9cefc4af-8f20-4aad-ba30-041f244f9740" Jul 17 21:17:55.713663 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:55.713634 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:55.787808 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:55.787781 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:55.916285 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:55.916256 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:56.022514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:56.022492 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:17:56.114298 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:56.114269 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:56.432182 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:56.432095 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:56.532608 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:56.532580 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:56.919915 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:56.919891 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:56.985257 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:56.985223 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:57.527583 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:57.527533 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:58.111743 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:58.111712 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:58.266663 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:58.266629 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:58.366842 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:58.366765 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:58.650982 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:58.650903 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:58.751877 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:58.751845 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:58.913245 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:58.913172 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7xx" Jul 17 21:17:58.913392 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:58.913344 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:59.341103 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:59.341063 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:17:59.944045 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:17:59.944012 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:00.520395 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:00.520368 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:00.693024 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:00.692985 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:18:00.695334 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:00.695315 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxrcv\" (UID: \"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f\") " pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:18:00.732938 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:00.732900 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:00.775816 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:00.775797 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rvj24\"" Jul 17 21:18:00.783485 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:00.783469 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jxrcv" Jul 17 21:18:00.946200 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:00.946174 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jxrcv"] Jul 17 21:18:00.949057 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:18:00.949026 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3b9fc0_328c_44c6_8efd_2c5a5bf6c70f.slice/crio-ca845bdc553307a8a4a576467781a5d076c9cf18b618220a9ee9ecf284a1df56 WatchSource:0}: Error finding container ca845bdc553307a8a4a576467781a5d076c9cf18b618220a9ee9ecf284a1df56: Status 404 returned error can't find the container with id ca845bdc553307a8a4a576467781a5d076c9cf18b618220a9ee9ecf284a1df56 Jul 17 21:18:01.042855 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:01.042783 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxrcv" event={"ID":"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f","Type":"ContainerStarted","Data":"ca845bdc553307a8a4a576467781a5d076c9cf18b618220a9ee9ecf284a1df56"} Jul 17 21:18:01.096515 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:01.096473 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:01.199299 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:01.199247 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:01.320361 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:01.320296 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:01.926581 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:01.926524 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:02.132492 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:02.132395 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:02.511762 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:02.511723 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:02.764608 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:02.764507 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:04.527971 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:04.527937 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:18:04.528442 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:04.528008 2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:18:04.528442 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:04.528407 2554 scope.go:117] "RemoveContainer" containerID="1f6ae1df854d0b245811494d7524b61b486b686f8d04e4fa11ef02e855605b82" Jul 17 21:18:04.528636 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:18:04.528613 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-qdvms_openshift-console-operator(9cefc4af-8f20-4aad-ba30-041f244f9740)\"" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" podUID="9cefc4af-8f20-4aad-ba30-041f244f9740" Jul 17 21:18:05.336121 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.336089 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:18:05.336306 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.336233 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:18:05.339277 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.339254 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 21:18:05.340302 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.340281 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 21:18:05.348956 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.348933 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6964db66-5d84-43e3-8f76-e7e23aeb63d3-original-pull-secret\") pod \"global-pull-secret-syncer-79tgn\" (UID: \"6964db66-5d84-43e3-8f76-e7e23aeb63d3\") " pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:18:05.349079 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.348958 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7507ea3a-57dc-4701-aaf0-5e241ab82b74-metrics-certs\") pod \"network-metrics-daemon-6w8mk\" (UID: \"7507ea3a-57dc-4701-aaf0-5e241ab82b74\") " pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:18:05.356963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.356942 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79tgn" Jul 17 21:18:05.436969 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.436930 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5bp\" (UniqueName: \"kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp\") pod \"network-check-target-7cbth\" (UID: \"98ba8f25-7586-4345-8e2e-4c87629466ca\") " pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:18:05.440078 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.440053 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt5bp\" (UniqueName: \"kubernetes.io/projected/98ba8f25-7586-4345-8e2e-4c87629466ca-kube-api-access-dt5bp\") pod \"network-check-target-7cbth\" (UID: \"98ba8f25-7586-4345-8e2e-4c87629466ca\") " pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:18:05.638859 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.638780 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lpmh2\"" Jul 17 21:18:05.646172 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.646148 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:18:05.650915 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.650892 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jrzlv\"" Jul 17 21:18:05.658581 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.658558 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6w8mk" Jul 17 21:18:05.980704 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.980675 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6w8mk"] Jul 17 21:18:05.984850 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:05.984824 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-79tgn"] Jul 17 21:18:05.985207 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:18:05.985188 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7507ea3a_57dc_4701_aaf0_5e241ab82b74.slice/crio-ca3533231f563312823365fa4a16ba59e6e073aacd742b10b2cc60c87edee54e WatchSource:0}: Error finding container ca3533231f563312823365fa4a16ba59e6e073aacd742b10b2cc60c87edee54e: Status 404 returned error can't find the container with id ca3533231f563312823365fa4a16ba59e6e073aacd742b10b2cc60c87edee54e Jul 17 21:18:05.987395 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:18:05.987368 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6964db66_5d84_43e3_8f76_e7e23aeb63d3.slice/crio-cb80b198b6623cd270214878e2e83be65d9d5cd6e82aa1b81c9fa032fd06f906 WatchSource:0}: Error finding container cb80b198b6623cd270214878e2e83be65d9d5cd6e82aa1b81c9fa032fd06f906: Status 404 returned error can't find the container with id cb80b198b6623cd270214878e2e83be65d9d5cd6e82aa1b81c9fa032fd06f906 Jul 17 21:18:06.011645 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.011619 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7cbth"] Jul 17 21:18:06.015270 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:18:06.015245 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ba8f25_7586_4345_8e2e_4c87629466ca.slice/crio-2934b57e61dbda0c00a593f4c94d6f929f5f880d7727ee4a7d32b58d9cf2f327 WatchSource:0}: Error finding container 2934b57e61dbda0c00a593f4c94d6f929f5f880d7727ee4a7d32b58d9cf2f327: Status 404 returned error can't find the container with id 2934b57e61dbda0c00a593f4c94d6f929f5f880d7727ee4a7d32b58d9cf2f327 Jul 17 21:18:06.041385 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.041364 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:18:06.041470 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.041403 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:18:06.041470 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.041443 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:18:06.042087 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.042068 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133e55a6-f3de-428f-a9c6-fa697726d63a-service-ca-bundle\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:18:06.043765 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.043746 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls\") pod \"image-registry-69f9779877-887qk\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:18:06.043876 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.043857 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133e55a6-f3de-428f-a9c6-fa697726d63a-metrics-certs\") pod \"router-default-74bdb64d9b-rxs6z\" (UID: \"133e55a6-f3de-428f-a9c6-fa697726d63a\") " pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:18:06.065217 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.065194 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-79tgn" event={"ID":"6964db66-5d84-43e3-8f76-e7e23aeb63d3","Type":"ContainerStarted","Data":"cb80b198b6623cd270214878e2e83be65d9d5cd6e82aa1b81c9fa032fd06f906"} Jul 17 21:18:06.066365 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.066324 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6w8mk" event={"ID":"7507ea3a-57dc-4701-aaf0-5e241ab82b74","Type":"ContainerStarted","Data":"ca3533231f563312823365fa4a16ba59e6e073aacd742b10b2cc60c87edee54e"} Jul 17 21:18:06.067587 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.067571 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxrcv" event={"ID":"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f","Type":"ContainerStarted","Data":"d3b4297d7f11b07ee383f60a25ddbb7c36fdbfa3358345859e414e082d2951ef"} Jul 17 21:18:06.068527 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.068487 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7cbth" event={"ID":"98ba8f25-7586-4345-8e2e-4c87629466ca","Type":"ContainerStarted","Data":"2934b57e61dbda0c00a593f4c94d6f929f5f880d7727ee4a7d32b58d9cf2f327"} Jul 17 21:18:06.142059 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.142021 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:18:06.142214 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.142070 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:18:06.142214 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.142102 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:18:06.144560 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.144482 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c76d2b-6ab9-421d-97af-7c9e03181c7c-metrics-tls\") pod \"dns-default-rvqhc\" (UID: \"17c76d2b-6ab9-421d-97af-7c9e03181c7c\") " pod="openshift-dns/dns-default-rvqhc" Jul 17 21:18:06.144660 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.144535 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec98f871-63b3-4790-bdcf-95a1409eb0c8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gjwmm\" (UID: \"ec98f871-63b3-4790-bdcf-95a1409eb0c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:18:06.144660 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.144597 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a491e347-ad20-4da2-97bb-ce0d561846bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-94h2v\" (UID: \"a491e347-ad20-4da2-97bb-ce0d561846bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:18:06.211304 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.211284 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fr9hx\"" Jul 17 21:18:06.219492 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.219468 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:18:06.265636 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.265604 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j2lmg\"" Jul 17 21:18:06.273240 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.273210 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:18:06.278475 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.278455 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wsskm\"" Jul 17 21:18:06.286875 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.286843 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" Jul 17 21:18:06.318107 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.317886 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lwcbh\"" Jul 17 21:18:06.324255 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.324222 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rvqhc" Jul 17 21:18:06.347821 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.347747 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mjxnm\"" Jul 17 21:18:06.355852 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.355824 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" Jul 17 21:18:06.363588 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.363422 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69f9779877-887qk"] Jul 17 21:18:06.448629 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.448462 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74bdb64d9b-rxs6z"] Jul 17 21:18:06.463643 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:18:06.463432 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133e55a6_f3de_428f_a9c6_fa697726d63a.slice/crio-45db2b3ce56d3d3475817ca1b0862f024dae58c3fced03c7ffaed44e66078858 WatchSource:0}: Error finding container 45db2b3ce56d3d3475817ca1b0862f024dae58c3fced03c7ffaed44e66078858: Status 404 returned error can't find the container with id 45db2b3ce56d3d3475817ca1b0862f024dae58c3fced03c7ffaed44e66078858 Jul 17 21:18:06.471055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.470899 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm"] Jul 17 21:18:06.539667 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.539571 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rvqhc"] Jul 17 21:18:06.548838 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:18:06.548718 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c76d2b_6ab9_421d_97af_7c9e03181c7c.slice/crio-48fbab82247c36bc7276695925ca47a78919fd8b9bd4b54f591851df35ddf6e1 WatchSource:0}: Error finding container 48fbab82247c36bc7276695925ca47a78919fd8b9bd4b54f591851df35ddf6e1: Status 404 returned error can't find the container with id 48fbab82247c36bc7276695925ca47a78919fd8b9bd4b54f591851df35ddf6e1 Jul 17 21:18:06.563566 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:06.563307 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v"] Jul 17 21:18:06.568937 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:18:06.568904 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda491e347_ad20_4da2_97bb_ce0d561846bb.slice/crio-7d63dace9504a0cf030b86cce833b18522b629bf3086b447a90cdcff7e6bc4a2 WatchSource:0}: Error finding container 7d63dace9504a0cf030b86cce833b18522b629bf3086b447a90cdcff7e6bc4a2: Status 404 returned error can't find the container with id 7d63dace9504a0cf030b86cce833b18522b629bf3086b447a90cdcff7e6bc4a2 Jul 17 21:18:07.078980 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.078949 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" event={"ID":"133e55a6-f3de-428f-a9c6-fa697726d63a","Type":"ContainerStarted","Data":"3d355ee985b1df4d21885b7c358f0b08cbc7100c87891bf9dbb76444bedaa780"} Jul 17 21:18:07.079313 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.079006 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" event={"ID":"133e55a6-f3de-428f-a9c6-fa697726d63a","Type":"ContainerStarted","Data":"45db2b3ce56d3d3475817ca1b0862f024dae58c3fced03c7ffaed44e66078858"} Jul 17 21:18:07.082098 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.082068 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" event={"ID":"ec98f871-63b3-4790-bdcf-95a1409eb0c8","Type":"ContainerStarted","Data":"8538f4dbbf300e3563df769044c82b227ab5df1758f7f8482627dc9a4340cd22"} Jul 17 21:18:07.085208 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.085139 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rvqhc" event={"ID":"17c76d2b-6ab9-421d-97af-7c9e03181c7c","Type":"ContainerStarted","Data":"48fbab82247c36bc7276695925ca47a78919fd8b9bd4b54f591851df35ddf6e1"} Jul 17 21:18:07.087845 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.087199 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f9779877-887qk" event={"ID":"97cee0f9-881c-44b3-9c70-8dc81c28699b","Type":"ContainerStarted","Data":"af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839"} Jul 17 21:18:07.087845 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.087242 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f9779877-887qk" event={"ID":"97cee0f9-881c-44b3-9c70-8dc81c28699b","Type":"ContainerStarted","Data":"dc4f841c3efcbd4f87b3176cda03592b360a34d7bf47ea85b772eda6e1305132"} Jul 17 21:18:07.087845 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.087771 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:18:07.090801 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.090324 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7cbth" event={"ID":"98ba8f25-7586-4345-8e2e-4c87629466ca","Type":"ContainerStarted","Data":"81677e1e92aecc594ed685a31efa128a6146ba7ae55a66be5af7982babd9990e"} Jul 17 21:18:07.090801 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.090768 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:18:07.094393 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.094356 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" event={"ID":"a491e347-ad20-4da2-97bb-ce0d561846bb","Type":"ContainerStarted","Data":"7d63dace9504a0cf030b86cce833b18522b629bf3086b447a90cdcff7e6bc4a2"} Jul 17 21:18:07.119422 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.118265 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" podStartSLOduration=38.118248364 podStartE2EDuration="38.118248364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:07.102007958 +0000 UTC m=+67.871355941" watchObservedRunningTime="2026-07-17 21:18:07.118248364 +0000 UTC m=+67.887596343" Jul 17 21:18:07.141560 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.140784 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7cbth" podStartSLOduration=67.140761497 podStartE2EDuration="1m7.140761497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:07.119289895 +0000 UTC m=+67.888637873" watchObservedRunningTime="2026-07-17 21:18:07.140761497 +0000 UTC m=+67.910109475" Jul 17 21:18:07.273885 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.273806 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:18:07.277404 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.277205 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:18:07.301193 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:07.299910 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69f9779877-887qk" podStartSLOduration=67.299889517 podStartE2EDuration="1m7.299889517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:07.141526877 +0000 UTC m=+67.910874855" watchObservedRunningTime="2026-07-17 21:18:07.299889517 +0000 UTC m=+68.069237497" Jul 17 21:18:08.109670 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:08.109211 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6w8mk" event={"ID":"7507ea3a-57dc-4701-aaf0-5e241ab82b74","Type":"ContainerStarted","Data":"1b3ca4efa1c2615fd8a91179b856043ba1d2da96da383f6865a3f23eeb77387c"} Jul 17 21:18:08.109670 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:08.109270 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6w8mk" event={"ID":"7507ea3a-57dc-4701-aaf0-5e241ab82b74","Type":"ContainerStarted","Data":"c6eb8db0a150a73eb16aca107cfbc3143c8f3e9a6304f8c8389f2ad729c76fe4"} Jul 17 21:18:08.109670 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:08.109650 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:18:08.110972 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:08.110911 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74bdb64d9b-rxs6z" Jul 17 21:18:08.129408 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:08.129362 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6w8mk" podStartSLOduration=66.707915084 podStartE2EDuration="1m8.12934515s" totalImagesPullingTime="1.421430066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:08.12924195 +0000 UTC m=+68.898589930" watchObservedRunningTime="2026-07-17 21:18:08.12934515 +0000 UTC m=+68.898693132" Jul 17 21:18:09.414378 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.414341 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff"] Jul 17 21:18:09.422730 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.422703 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.427563 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.427517 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 17 21:18:09.428085 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.427783 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 17 21:18:09.428085 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.428017 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 17 21:18:09.429534 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.428910 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 17 21:18:09.429534 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.429136 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 17 21:18:09.429534 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.429310 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 17 21:18:09.429534 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.429509 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 17 21:18:09.437731 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.436012 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff"] Jul 17 21:18:09.576714 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.576675 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-ca\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.576877 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.576752 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-hub\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.576877 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.576781 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/570de211-3d6d-4e32-9f4e-614950e78178-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.576877 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.576821 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.577007 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.576942 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.577007 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.576972 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9nlf\" (UniqueName: \"kubernetes.io/projected/570de211-3d6d-4e32-9f4e-614950e78178-kube-api-access-g9nlf\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.584131 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.584071 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:09.678248 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.678164 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.678248 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.678202 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9nlf\" (UniqueName: \"kubernetes.io/projected/570de211-3d6d-4e32-9f4e-614950e78178-kube-api-access-g9nlf\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.678248 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.678233 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-ca\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.678514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.678274 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-hub\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.678514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.678295 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/570de211-3d6d-4e32-9f4e-614950e78178-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.678514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.678323 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.679207 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.679150 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/570de211-3d6d-4e32-9f4e-614950e78178-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.681238 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.681212 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-hub\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.681374 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.681216 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.681374 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.681333 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.681484 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.681450 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/570de211-3d6d-4e32-9f4e-614950e78178-ca\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.683975 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.683950 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:09.690661 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.690616 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9nlf\" (UniqueName: \"kubernetes.io/projected/570de211-3d6d-4e32-9f4e-614950e78178-kube-api-access-g9nlf\") pod \"cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff\" (UID: \"570de211-3d6d-4e32-9f4e-614950e78178\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.750107 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.750073 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" Jul 17 21:18:09.825832 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.825789 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:09.976132 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.976021 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:09.980566 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.980521 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:18:09.983528 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:09.983506 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vt6tr\" (UID: \"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:18:10.134758 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:10.134727 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8c2tv\"" Jul 17 21:18:10.141424 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:10.141397 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" Jul 17 21:18:10.989284 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:10.989238 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:11.127937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:11.127907 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:12.988076 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:12.988001 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:13.050328 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:13.050279 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:13.283775 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:13.282058 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:13.385251 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:13.385082 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff"] Jul 17 21:18:13.387238 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:13.387215 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:13.418289 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:13.416201 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr"] Jul 17 21:18:13.426163 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:18:13.426049 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2d76be_257f_4a0e_9e8b_45ed8cf0ea71.slice/crio-60f9a73bd3d72f892f201da40cbf7bfb8b35ff6a33234492e1b5a71dd0e643ab WatchSource:0}: Error finding container 60f9a73bd3d72f892f201da40cbf7bfb8b35ff6a33234492e1b5a71dd0e643ab: Status 404 returned error can't find the container with id 60f9a73bd3d72f892f201da40cbf7bfb8b35ff6a33234492e1b5a71dd0e643ab Jul 17 21:18:13.721344 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:13.721318 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:13.831802 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:13.831776 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:13.941445 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:13.941420 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:14.043212 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.043181 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:14.136898 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.136816 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" event={"ID":"ec98f871-63b3-4790-bdcf-95a1409eb0c8","Type":"ContainerStarted","Data":"133d6894181276a1586a2d69567746824f40d91eeecf87835fc7ee8e93f1883c"} Jul 17 21:18:14.136898 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.136881 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" event={"ID":"ec98f871-63b3-4790-bdcf-95a1409eb0c8","Type":"ContainerStarted","Data":"5b5f6fb7d5b936ddd803324beeb9c585036133aabb17a846a388047d44f7b31b"} Jul 17 21:18:14.139063 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.139038 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxrcv" event={"ID":"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f","Type":"ContainerStarted","Data":"70f492f20858c1cbc5dc8aa46fa0435b4fec4ff39cfdf2037234cec1b06648d0"} Jul 17 21:18:14.140600 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.140573 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" event={"ID":"a491e347-ad20-4da2-97bb-ce0d561846bb","Type":"ContainerStarted","Data":"210e95f632d3f00a3b8b06cc05d2e5644d16fa8f5c02666f1f669019c26499cf"} Jul 17 21:18:14.141906 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.141879 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" event={"ID":"570de211-3d6d-4e32-9f4e-614950e78178","Type":"ContainerStarted","Data":"a855ebbe16afa7623077dba781e1d46a5a9f1293c92495ac2692f8f3c497e438"} Jul 17 21:18:14.143419 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.143396 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-79tgn" event={"ID":"6964db66-5d84-43e3-8f76-e7e23aeb63d3","Type":"ContainerStarted","Data":"3a68a88198ff35ddf57c6e74afb0fdb69ec54c7f22de669e35ee95d3a387adfa"} Jul 17 21:18:14.144593 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.144568 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" event={"ID":"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71","Type":"ContainerStarted","Data":"60f9a73bd3d72f892f201da40cbf7bfb8b35ff6a33234492e1b5a71dd0e643ab"} Jul 17 21:18:14.146445 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.146425 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rvqhc" event={"ID":"17c76d2b-6ab9-421d-97af-7c9e03181c7c","Type":"ContainerStarted","Data":"30263d377c24cd4acb66b3ae0f311bbbe5d40b94d1b417c2ecc85f2a0545eefe"} Jul 17 21:18:14.146535 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.146467 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rvqhc" event={"ID":"17c76d2b-6ab9-421d-97af-7c9e03181c7c","Type":"ContainerStarted","Data":"3b1351530bf2c61b9de43bc83d9ca8f013196606f9a5cfd73b5da06b8e9db2fa"} Jul 17 21:18:14.146618 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.146600 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rvqhc" Jul 17 21:18:14.165128 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.165084 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gjwmm" podStartSLOduration=38.500942238 podStartE2EDuration="45.165066568s" totalImagesPullingTime="6.66412433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:14.163927301 +0000 UTC m=+74.933275281" watchObservedRunningTime="2026-07-17 21:18:14.165066568 +0000 UTC m=+74.934414552" Jul 17 21:18:14.197447 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.197410 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-79tgn" podStartSLOduration=66.966075547 podStartE2EDuration="1m14.197396539s" totalImagesPullingTime="7.231320992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:14.196194922 +0000 UTC m=+74.965542899" watchObservedRunningTime="2026-07-17 21:18:14.197396539 +0000 UTC m=+74.966744547" Jul 17 21:18:14.361892 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.361830 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rvqhc" podStartSLOduration=33.714445385 podStartE2EDuration="40.361806975s" totalImagesPullingTime="6.64736159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:14.286069824 +0000 UTC m=+75.055417815" watchObservedRunningTime="2026-07-17 21:18:14.361806975 +0000 UTC m=+75.131154956" Jul 17 21:18:14.362418 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.362375 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-94h2v" podStartSLOduration=37.73769682 podStartE2EDuration="44.36236527s" totalImagesPullingTime="6.62466845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:14.36037983 +0000 UTC m=+75.129727810" watchObservedRunningTime="2026-07-17 21:18:14.36236527 +0000 UTC m=+75.131713251" Jul 17 21:18:14.714458 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:14.714426 2554 scope.go:117] "RemoveContainer" containerID="1f6ae1df854d0b245811494d7524b61b486b686f8d04e4fa11ef02e855605b82" Jul 17 21:18:15.152843 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:15.152812 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" event={"ID":"dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71","Type":"ContainerStarted","Data":"d990b54e2ce7b6d88a5f8009e836e7960481150ba9b0a597d6bde6b852f34593"} Jul 17 21:18:15.154944 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:15.154922 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:18:15.155073 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:15.154982 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" event={"ID":"9cefc4af-8f20-4aad-ba30-041f244f9740","Type":"ContainerStarted","Data":"d9356145b52f46a09a3fc7b06505099d93ac70ff71867d90b49e253006734ec8"} Jul 17 21:18:15.155809 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:15.155714 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:18:15.172334 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:15.172284 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vt6tr" podStartSLOduration=36.691501761 podStartE2EDuration="38.172251896s" totalImagesPullingTime="1.480750135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:15.170063646 +0000 UTC m=+75.939411624" watchObservedRunningTime="2026-07-17 21:18:15.172251896 +0000 UTC m=+75.941599875" Jul 17 21:18:15.190282 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:15.190085 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" podStartSLOduration=37.571756583 podStartE2EDuration="46.190071267s" totalImagesPullingTime="8.618314684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:15.1897015 +0000 UTC m=+75.959049479" watchObservedRunningTime="2026-07-17 21:18:15.190071267 +0000 UTC m=+75.959419238" Jul 17 21:18:15.312281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:15.312247 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:15.379536 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:15.379495 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:15.545054 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:15.545013 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-qdvms" Jul 17 21:18:15.773255 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:15.773123 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:15.839532 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:15.839494 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:16.049296 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:16.049213 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:16.119261 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:16.119222 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:16.177959 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:16.177921 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:16.233487 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:16.233453 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:16.414301 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:16.414220 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:16.614052 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:16.614022 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:16.691422 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:16.691333 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:17.626365 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:17.626336 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:17.827400 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:17.827361 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:18.039324 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:18.039295 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:18.096331 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:18.096299 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:18.167685 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:18.167653 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxrcv" event={"ID":"fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f","Type":"ContainerStarted","Data":"098f06e1124e015c6b6cbaee790ea544409ea5adb79a84b9521961fd26b34f18"} Jul 17 21:18:18.188538 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:18.188482 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jxrcv" podStartSLOduration=18.257415311 podStartE2EDuration="34.188467653s" totalImagesPullingTime="15.931052342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 21:17:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:18.18672649 +0000 UTC m=+78.956074469" watchObservedRunningTime="2026-07-17 21:18:18.188467653 +0000 UTC m=+78.957815630" Jul 17 21:18:18.286242 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:18.286209 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:19.923752 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:19.923715 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:20.005191 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:20.005158 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:20.368178 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:20.368140 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:23.943209 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:23.943174 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:24.157282 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:24.157257 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rvqhc" Jul 17 21:18:24.234803 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:24.234714 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:24.293090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:24.293067 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:25.996696 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:25.996664 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:26.096027 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.095998 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:26.197540 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.197502 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:26.224359 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.224333 2554 patch_prober.go:28] interesting pod/image-registry-69f9779877-887qk container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 17 21:18:26.224481 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.224375 2554 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-69f9779877-887qk" podUID="97cee0f9-881c-44b3-9c70-8dc81c28699b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 21:18:26.332814 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.332784 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:26.394374 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.394330 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tc4x7"] Jul 17 21:18:26.400852 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.400826 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.403779 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.403759 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 17 21:18:26.404318 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.404297 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p8ktt\"" Jul 17 21:18:26.404730 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.404717 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 17 21:18:26.405312 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.405295 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 17 21:18:26.405512 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.405496 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 17 21:18:26.432920 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.432892 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:26.513680 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.513651 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9218f58d-90fe-42fe-a695-60cee3beb9ca-metrics-client-ca\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.513829 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.513706 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-wtmp\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.513829 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.513731 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.513829 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.513769 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-tls\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.513936 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.513839 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65c7g\" (UniqueName: \"kubernetes.io/projected/9218f58d-90fe-42fe-a695-60cee3beb9ca-kube-api-access-65c7g\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.513936 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.513888 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9218f58d-90fe-42fe-a695-60cee3beb9ca-root\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.513936 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.513915 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-textfile\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.514025 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.513946 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9218f58d-90fe-42fe-a695-60cee3beb9ca-sys\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.514025 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.513969 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-accelerators-collector-config\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.532038 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.532016 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:26.615256 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615173 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65c7g\" (UniqueName: \"kubernetes.io/projected/9218f58d-90fe-42fe-a695-60cee3beb9ca-kube-api-access-65c7g\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615256 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615205 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9218f58d-90fe-42fe-a695-60cee3beb9ca-root\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615256 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615228 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-textfile\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615256 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615256 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9218f58d-90fe-42fe-a695-60cee3beb9ca-sys\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615572 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615281 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-accelerators-collector-config\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615572 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615305 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9218f58d-90fe-42fe-a695-60cee3beb9ca-root\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615572 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615326 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9218f58d-90fe-42fe-a695-60cee3beb9ca-metrics-client-ca\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615572 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615338 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9218f58d-90fe-42fe-a695-60cee3beb9ca-sys\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615572 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615363 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-wtmp\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615572 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615400 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615572 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615441 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-tls\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615572 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615483 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-wtmp\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615959 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615638 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-textfile\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.615959 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615906 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9218f58d-90fe-42fe-a695-60cee3beb9ca-metrics-client-ca\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.616039 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.615975 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-accelerators-collector-config\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.617860 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.617840 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.617952 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.617932 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9218f58d-90fe-42fe-a695-60cee3beb9ca-node-exporter-tls\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.624992 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.624971 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65c7g\" (UniqueName: \"kubernetes.io/projected/9218f58d-90fe-42fe-a695-60cee3beb9ca-kube-api-access-65c7g\") pod \"node-exporter-tc4x7\" (UID: \"9218f58d-90fe-42fe-a695-60cee3beb9ca\") " pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.664165 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.664134 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:26.709909 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.709881 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tc4x7" Jul 17 21:18:26.721804 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:18:26.721775 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9218f58d_90fe_42fe_a695_60cee3beb9ca.slice/crio-90d1cbac5d0d422e8f78d5e71f44fc3f7b72dfd651fd1ebfc3264575d88193bb WatchSource:0}: Error finding container 90d1cbac5d0d422e8f78d5e71f44fc3f7b72dfd651fd1ebfc3264575d88193bb: Status 404 returned error can't find the container with id 90d1cbac5d0d422e8f78d5e71f44fc3f7b72dfd651fd1ebfc3264575d88193bb Jul 17 21:18:26.762739 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.762712 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:26.905343 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:26.905261 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:27.005375 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:27.005342 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:27.202360 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:27.202287 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tc4x7" event={"ID":"9218f58d-90fe-42fe-a695-60cee3beb9ca","Type":"ContainerStarted","Data":"90d1cbac5d0d422e8f78d5e71f44fc3f7b72dfd651fd1ebfc3264575d88193bb"} Jul 17 21:18:27.333509 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:27.333474 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:27.424538 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:27.424488 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:27.594393 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:27.594341 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:28.209159 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:28.209114 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tc4x7" event={"ID":"9218f58d-90fe-42fe-a695-60cee3beb9ca","Type":"ContainerStarted","Data":"80881b3da88d762922dc8b07e9efd55048af4410642820961becf84676466b61"} Jul 17 21:18:29.119811 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:29.119776 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:18:29.215464 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:29.215329 2554 generic.go:356] "Generic (PLEG): container finished" podID="9218f58d-90fe-42fe-a695-60cee3beb9ca" containerID="80881b3da88d762922dc8b07e9efd55048af4410642820961becf84676466b61" exitCode=0 Jul 17 21:18:29.215464 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:29.215410 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tc4x7" event={"ID":"9218f58d-90fe-42fe-a695-60cee3beb9ca","Type":"ContainerDied","Data":"80881b3da88d762922dc8b07e9efd55048af4410642820961becf84676466b61"} Jul 17 21:18:29.475810 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:29.475729 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:29.593686 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:29.593647 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:29.690128 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:29.690096 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:29.796409 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:29.796380 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:30.856656 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:30.856615 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:30.955867 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:30.955837 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:31.137904 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.137828 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:31.225020 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.224978 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" event={"ID":"570de211-3d6d-4e32-9f4e-614950e78178","Type":"ContainerStarted","Data":"0cdd49f075291a125cd2227b89ddb8cc52981758a2d7e3bef2c245ed48e15867"} Jul 17 21:18:31.225020 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.225025 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" event={"ID":"570de211-3d6d-4e32-9f4e-614950e78178","Type":"ContainerStarted","Data":"f65a70c89ec40f132dbbf27dadeaccda310587ebf0b8bd6ee1832b2e5e3d9757"} Jul 17 21:18:31.225282 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.225045 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" event={"ID":"570de211-3d6d-4e32-9f4e-614950e78178","Type":"ContainerStarted","Data":"42eed9bf0387515b01cd9cfd3278a9cfefb881f65e417aba60f1c4fd671ff5fb"} Jul 17 21:18:31.227245 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.227224 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tc4x7" event={"ID":"9218f58d-90fe-42fe-a695-60cee3beb9ca","Type":"ContainerStarted","Data":"d1cac1b93d42772733f03d3486f243d875d0172eb54718e4de9a5519a624d20f"} Jul 17 21:18:31.227335 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.227286 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tc4x7" event={"ID":"9218f58d-90fe-42fe-a695-60cee3beb9ca","Type":"ContainerStarted","Data":"2810bc90355641623efec0550337b8e1335dd749a2f458f6c0f74b544017d22c"} Jul 17 21:18:31.251449 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.251411 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55d6cf5cd6-qn5ff" podStartSLOduration=5.383437446 podStartE2EDuration="22.251398376s" totalImagesPullingTime="16.86796093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:18:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:31.249384422 +0000 UTC m=+92.018732400" watchObservedRunningTime="2026-07-17 21:18:31.251398376 +0000 UTC m=+92.020746353" Jul 17 21:18:31.252237 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.252215 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:31.274846 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.274807 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tc4x7" podStartSLOduration=3.918124879 podStartE2EDuration="5.274794786s" totalImagesPullingTime="1.26622213s" totalInitContainerRuntime="90.447777ms" isStatefulPod=true podCreationTimestamp="2026-07-17 21:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:18:31.273089898 +0000 UTC m=+92.042437876" watchObservedRunningTime="2026-07-17 21:18:31.274794786 +0000 UTC m=+92.044142763" Jul 17 21:18:31.354810 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.354786 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:31.896205 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.896176 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69f9779877-887qk"] Jul 17 21:18:31.994473 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:31.994434 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:32.095149 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:32.095120 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:32.196216 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:32.196127 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:32.684968 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:32.684930 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:33.270959 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:33.270923 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:33.372354 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:33.372315 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:36.708564 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:36.708515 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:36.773118 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:36.773088 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:38.073535 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:38.073504 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:38.174507 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:38.174477 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:38.251401 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:38.251370 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:39.117164 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:39.117135 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7cbth" Jul 17 21:18:41.923537 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:41.923500 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:42.122275 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:42.122240 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:42.520839 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:42.520807 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:42.658260 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:42.658227 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:43.271801 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:43.271763 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:43.420224 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:43.420190 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:45.241870 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:45.241828 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:45.357853 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:45.357812 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:45.717729 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:45.717697 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:46.120518 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:46.120483 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:46.195943 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:46.195906 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:49.625780 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:49.625746 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:49.729242 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:49.729205 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:55.324342 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:55.324312 2554 generic.go:356] "Generic (PLEG): container finished" podID="999ecc4f-66b1-4ecb-97c5-c54040757d19" containerID="a0ba0873ca6185efad43c7e620dcdf567ba20945cc4f3cf59b9d9dee03c98b36" exitCode=0 Jul 17 21:18:55.324847 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:55.324393 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-k9wjf" event={"ID":"999ecc4f-66b1-4ecb-97c5-c54040757d19","Type":"ContainerDied","Data":"a0ba0873ca6185efad43c7e620dcdf567ba20945cc4f3cf59b9d9dee03c98b36"} Jul 17 21:18:55.324847 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:55.324821 2554 scope.go:117] "RemoveContainer" containerID="a0ba0873ca6185efad43c7e620dcdf567ba20945cc4f3cf59b9d9dee03c98b36" Jul 17 21:18:56.329514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:56.329480 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-k9wjf" event={"ID":"999ecc4f-66b1-4ecb-97c5-c54040757d19","Type":"ContainerStarted","Data":"47a6d0950f71d72ffcb7dd0459018e65baf7ca2dc0f611c279c425a66dd3933c"} Jul 17 21:18:56.916145 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:56.916108 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69f9779877-887qk" podUID="97cee0f9-881c-44b3-9c70-8dc81c28699b" containerName="registry" containerID="cri-o://af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839" gracePeriod=30 Jul 17 21:18:57.169863 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.169811 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:18:57.239930 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.239897 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-image-registry-private-configuration\") pod \"97cee0f9-881c-44b3-9c70-8dc81c28699b\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " Jul 17 21:18:57.240105 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.239949 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-certificates\") pod \"97cee0f9-881c-44b3-9c70-8dc81c28699b\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " Jul 17 21:18:57.240105 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.239968 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-bound-sa-token\") pod \"97cee0f9-881c-44b3-9c70-8dc81c28699b\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " Jul 17 21:18:57.240105 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.240028 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-trusted-ca\") pod \"97cee0f9-881c-44b3-9c70-8dc81c28699b\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " Jul 17 21:18:57.240105 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.240052 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlcpd\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-kube-api-access-vlcpd\") pod \"97cee0f9-881c-44b3-9c70-8dc81c28699b\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " Jul 17 21:18:57.240105 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.240081 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls\") pod \"97cee0f9-881c-44b3-9c70-8dc81c28699b\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " Jul 17 21:18:57.240365 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.240108 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-installation-pull-secrets\") pod \"97cee0f9-881c-44b3-9c70-8dc81c28699b\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " Jul 17 21:18:57.240365 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.240140 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97cee0f9-881c-44b3-9c70-8dc81c28699b-ca-trust-extracted\") pod \"97cee0f9-881c-44b3-9c70-8dc81c28699b\" (UID: \"97cee0f9-881c-44b3-9c70-8dc81c28699b\") " Jul 17 21:18:57.240469 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.240368 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "97cee0f9-881c-44b3-9c70-8dc81c28699b" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 21:18:57.240657 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.240597 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "97cee0f9-881c-44b3-9c70-8dc81c28699b" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 21:18:57.242702 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.242671 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "97cee0f9-881c-44b3-9c70-8dc81c28699b" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 21:18:57.242815 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.242789 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "97cee0f9-881c-44b3-9c70-8dc81c28699b" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 21:18:57.242874 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.242820 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "97cee0f9-881c-44b3-9c70-8dc81c28699b" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 21:18:57.243006 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.242979 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-kube-api-access-vlcpd" (OuterVolumeSpecName: "kube-api-access-vlcpd") pod "97cee0f9-881c-44b3-9c70-8dc81c28699b" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b"). InnerVolumeSpecName "kube-api-access-vlcpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 21:18:57.243113 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.243090 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "97cee0f9-881c-44b3-9c70-8dc81c28699b" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 21:18:57.248753 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.248729 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97cee0f9-881c-44b3-9c70-8dc81c28699b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "97cee0f9-881c-44b3-9c70-8dc81c28699b" (UID: "97cee0f9-881c-44b3-9c70-8dc81c28699b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 21:18:57.334882 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.334847 2554 generic.go:356] "Generic (PLEG): container finished" podID="97cee0f9-881c-44b3-9c70-8dc81c28699b" containerID="af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839" exitCode=0 Jul 17 21:18:57.335349 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.334899 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f9779877-887qk" event={"ID":"97cee0f9-881c-44b3-9c70-8dc81c28699b","Type":"ContainerDied","Data":"af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839"} Jul 17 21:18:57.335349 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.334905 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f9779877-887qk" Jul 17 21:18:57.335349 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.334938 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f9779877-887qk" event={"ID":"97cee0f9-881c-44b3-9c70-8dc81c28699b","Type":"ContainerDied","Data":"dc4f841c3efcbd4f87b3176cda03592b360a34d7bf47ea85b772eda6e1305132"} Jul 17 21:18:57.335349 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.334960 2554 scope.go:117] "RemoveContainer" containerID="af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839" Jul 17 21:18:57.341373 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.341335 2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-certificates\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:18:57.341373 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.341370 2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-bound-sa-token\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:18:57.341586 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.341386 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97cee0f9-881c-44b3-9c70-8dc81c28699b-trusted-ca\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:18:57.341586 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.341400 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlcpd\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-kube-api-access-vlcpd\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:18:57.341586 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.341412 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97cee0f9-881c-44b3-9c70-8dc81c28699b-registry-tls\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:18:57.341586 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.341425 2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-installation-pull-secrets\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:18:57.341586 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.341438 2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97cee0f9-881c-44b3-9c70-8dc81c28699b-ca-trust-extracted\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:18:57.341586 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.341453 2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97cee0f9-881c-44b3-9c70-8dc81c28699b-image-registry-private-configuration\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:18:57.343733 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.343711 2554 scope.go:117] "RemoveContainer" containerID="af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839" Jul 17 21:18:57.343973 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:18:57.343953 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839\": container with ID starting with af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839 not found: ID does not exist" containerID="af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839" Jul 17 21:18:57.344018 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.343980 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839"} err="failed to get container status \"af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839\": rpc error: code = NotFound desc = could not find container \"af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839\": container with ID starting with af3533ce4548cde51b505c28d9d9b0cbda59cc290bc272acd4552c9c81aa1839 not found: ID does not exist" Jul 17 21:18:57.357025 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.357003 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69f9779877-887qk"] Jul 17 21:18:57.360636 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.360609 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69f9779877-887qk"] Jul 17 21:18:57.718343 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:57.718312 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97cee0f9-881c-44b3-9c70-8dc81c28699b" path="/var/lib/kubelet/pods/97cee0f9-881c-44b3-9c70-8dc81c28699b/volumes" Jul 17 21:18:58.262236 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:58.262185 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:18:58.326279 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:18:58.326248 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:19:03.807838 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:03.807421 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:19:03.879997 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:03.879946 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:19:09.383753 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:09.383723 2554 generic.go:356] "Generic (PLEG): container finished" podID="036870c8-94b9-4906-8b4a-ea70695c9c79" containerID="3bb22d5616f92235e56b077cae62acc8bea120e6fb984e8bb52e3993d8f82036" exitCode=0 Jul 17 21:19:09.384202 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:09.383800 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" event={"ID":"036870c8-94b9-4906-8b4a-ea70695c9c79","Type":"ContainerDied","Data":"3bb22d5616f92235e56b077cae62acc8bea120e6fb984e8bb52e3993d8f82036"} Jul 17 21:19:09.384202 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:09.384146 2554 scope.go:117] "RemoveContainer" containerID="3bb22d5616f92235e56b077cae62acc8bea120e6fb984e8bb52e3993d8f82036" Jul 17 21:19:09.385458 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:09.385436 2554 generic.go:356] "Generic (PLEG): container finished" podID="fbe62ded-80ca-4471-ae23-dc1abdce5895" containerID="f3677263abcb87e9716f741c221449aeabeff5e5f6dcdbeb7cac982c6e790e94" exitCode=0 Jul 17 21:19:09.385582 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:09.385485 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" event={"ID":"fbe62ded-80ca-4471-ae23-dc1abdce5895","Type":"ContainerDied","Data":"f3677263abcb87e9716f741c221449aeabeff5e5f6dcdbeb7cac982c6e790e94"} Jul 17 21:19:09.385841 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:09.385827 2554 scope.go:117] "RemoveContainer" containerID="f3677263abcb87e9716f741c221449aeabeff5e5f6dcdbeb7cac982c6e790e94" Jul 17 21:19:10.390827 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:10.390781 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w8cbj" event={"ID":"036870c8-94b9-4906-8b4a-ea70695c9c79","Type":"ContainerStarted","Data":"29def0be5db4f771043bc6c456ef769970c90e6ecbb05bc37dcd5593823d93cd"} Jul 17 21:19:10.392536 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:10.392513 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qhb42" event={"ID":"fbe62ded-80ca-4471-ae23-dc1abdce5895","Type":"ContainerStarted","Data":"5496c29c87b53f4505e99b5cf2502c46145201ad8d1610d85bf75c9c3e51827e"} Jul 17 21:19:42.287676 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:42.287624 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:19:45.712890 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:45.712849 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:19:46.020197 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:46.020098 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:19:51.145401 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:51.145301 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:19:54.595792 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:19:54.595757 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:20:05.525103 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:20:05.525065 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:20:05.706813 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:20:05.706776 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:20:26.602250 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:20:26.602211 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:20:26.712500 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:20:26.712461 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:01.166898 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:01.166847 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:01.308071 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:01.308034 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:13.090959 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:13.090924 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:13.147888 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:13.147858 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:16.012599 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:16.012538 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:26.720782 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:26.720690 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:26.781600 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:26.781535 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:26.848033 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:26.847988 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:28.274109 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:28.274060 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:28.373560 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:28.373510 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:38.235875 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:38.235836 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:57.144121 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:57.144088 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:21:59.562104 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:59.562077 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:21:59.572402 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:59.572377 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:21:59.572529 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:21:59.572405 2554 kubelet.go:1648] "Image garbage collection succeeded" Jul 17 21:22:10.220209 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:10.220173 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:11.459364 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:11.459326 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:32.418417 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:32.418375 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:33.415958 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:33.415923 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:36.045705 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:36.045671 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:36.176535 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:36.176492 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:39.196246 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:39.196207 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:39.371579 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:39.371524 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:41.530934 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:41.530901 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:41.741976 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:41.741948 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:42.417485 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:42.417450 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:42.482416 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:42.482386 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:49.143574 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:49.143526 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:50.390337 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:50.390300 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:50.482460 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:50.482385 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:57.127116 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:57.127076 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:57.226337 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:57.226279 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:22:59.342853 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:22:59.342817 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:04.815329 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:04.815288 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:04.916079 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:04.916044 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:14.738508 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:14.738470 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:14.836753 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:14.836715 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:15.353126 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.353076 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:15.666968 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.666867 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:15.769451 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.769424 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:15.785609 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.785584 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw"] Jul 17 21:23:15.786168 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.786148 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97cee0f9-881c-44b3-9c70-8dc81c28699b" containerName="registry" Jul 17 21:23:15.786315 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.786303 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="97cee0f9-881c-44b3-9c70-8dc81c28699b" containerName="registry" Jul 17 21:23:15.786464 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.786454 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="97cee0f9-881c-44b3-9c70-8dc81c28699b" containerName="registry" Jul 17 21:23:15.789767 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.789747 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.792822 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.792801 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 17 21:23:15.792933 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.792827 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x6d45\"" Jul 17 21:23:15.793070 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.793053 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 17 21:23:15.793291 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.793275 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 17 21:23:15.793616 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.793599 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 17 21:23:15.818782 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.818762 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw"] Jul 17 21:23:15.835883 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.835861 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c3c450c-8ea9-4253-8733-ab4ca75cc368-tmp\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.835975 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.835900 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfg5q\" (UniqueName: \"kubernetes.io/projected/6c3c450c-8ea9-4253-8733-ab4ca75cc368-kube-api-access-qfg5q\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.835975 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.835927 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c3c450c-8ea9-4253-8733-ab4ca75cc368-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.836055 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.835995 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c3c450c-8ea9-4253-8733-ab4ca75cc368-manifests\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.836102 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.836071 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c3c450c-8ea9-4253-8733-ab4ca75cc368-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.936849 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.936778 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfg5q\" (UniqueName: \"kubernetes.io/projected/6c3c450c-8ea9-4253-8733-ab4ca75cc368-kube-api-access-qfg5q\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.936849 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.936817 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c3c450c-8ea9-4253-8733-ab4ca75cc368-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.936849 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.936848 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c3c450c-8ea9-4253-8733-ab4ca75cc368-manifests\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.937042 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.936879 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c3c450c-8ea9-4253-8733-ab4ca75cc368-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.937042 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.936906 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c3c450c-8ea9-4253-8733-ab4ca75cc368-tmp\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.937371 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.937349 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c3c450c-8ea9-4253-8733-ab4ca75cc368-manifests\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.939306 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.939284 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c3c450c-8ea9-4253-8733-ab4ca75cc368-tmp\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.939534 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.939515 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c3c450c-8ea9-4253-8733-ab4ca75cc368-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.939627 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.939530 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c3c450c-8ea9-4253-8733-ab4ca75cc368-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:15.951317 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:15.951296 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfg5q\" (UniqueName: \"kubernetes.io/projected/6c3c450c-8ea9-4253-8733-ab4ca75cc368-kube-api-access-qfg5q\") pod \"opendatahub-operator-controller-manager-6d6dbd9688-zk9jw\" (UID: \"6c3c450c-8ea9-4253-8733-ab4ca75cc368\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:16.031020 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:16.030967 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:16.099561 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:16.099519 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:16.108939 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:16.108885 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:16.215068 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:16.214988 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:16.238197 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:16.238174 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw"] Jul 17 21:23:16.240998 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:23:16.240973 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c3c450c_8ea9_4253_8733_ab4ca75cc368.slice/crio-34d56fa06483064e224019b17e9c6a6b5ca7b4bb2a98a67d3d0af37523c8ed6d WatchSource:0}: Error finding container 34d56fa06483064e224019b17e9c6a6b5ca7b4bb2a98a67d3d0af37523c8ed6d: Status 404 returned error can't find the container with id 34d56fa06483064e224019b17e9c6a6b5ca7b4bb2a98a67d3d0af37523c8ed6d Jul 17 21:23:16.243287 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:16.243271 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 21:23:16.303834 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:16.303809 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:16.368717 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:16.368690 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:17.251187 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:17.251152 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" event={"ID":"6c3c450c-8ea9-4253-8733-ab4ca75cc368","Type":"ContainerStarted","Data":"34d56fa06483064e224019b17e9c6a6b5ca7b4bb2a98a67d3d0af37523c8ed6d"} Jul 17 21:23:20.263147 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:20.263108 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" event={"ID":"6c3c450c-8ea9-4253-8733-ab4ca75cc368","Type":"ContainerStarted","Data":"ef4bfa11fc93883e8b73d1bcce9279c82e61b15d66592f4996d095b4dad914ff"} Jul 17 21:23:21.268032 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:21.268001 2554 generic.go:356] "Generic (PLEG): container finished" podID="6c3c450c-8ea9-4253-8733-ab4ca75cc368" containerID="ef4bfa11fc93883e8b73d1bcce9279c82e61b15d66592f4996d095b4dad914ff" exitCode=0 Jul 17 21:23:21.268032 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:21.268036 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" event={"ID":"6c3c450c-8ea9-4253-8733-ab4ca75cc368","Type":"ContainerDied","Data":"ef4bfa11fc93883e8b73d1bcce9279c82e61b15d66592f4996d095b4dad914ff"} Jul 17 21:23:22.273563 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:22.273515 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" event={"ID":"6c3c450c-8ea9-4253-8733-ab4ca75cc368","Type":"ContainerStarted","Data":"5581f13a68399c3d395e5a5f50390c65d4e708222211804342b98522a4e6a0d5"} Jul 17 21:23:22.273928 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:22.273640 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:22.307809 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:22.307764 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" podStartSLOduration=2.87526686 podStartE2EDuration="7.307747909s" totalImagesPullingTime="3.414547625s" totalInitContainerRuntime="1.017933424s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:23:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:23:22.307449107 +0000 UTC m=+383.076797086" watchObservedRunningTime="2026-07-17 21:23:22.307747909 +0000 UTC m=+383.077095886" Jul 17 21:23:26.551413 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:26.551377 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:26.617220 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:26.617181 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:26.986321 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:26.986236 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:27.062490 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:27.062458 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:28.365080 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:28.365047 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:28.453844 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:28.453807 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:31.871399 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:31.871359 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:31.974611 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:31.974575 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:33.279816 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:33.279791 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d6dbd9688-zk9jw" Jul 17 21:23:35.061609 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.061566 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:35.246475 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.246443 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:35.739228 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.739184 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:35.802070 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.802039 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw"] Jul 17 21:23:35.810353 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.810334 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" Jul 17 21:23:35.813425 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.813404 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 17 21:23:35.813571 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.813526 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 17 21:23:35.813697 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.813682 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8s8c5\"" Jul 17 21:23:35.840937 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.840913 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw"] Jul 17 21:23:35.895490 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.895459 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6776k\" (UniqueName: \"kubernetes.io/projected/62f4330c-55ea-46a3-9027-f1723405db8e-kube-api-access-6776k\") pod \"kube-auth-proxy-5fbb8dc498-xsnxw\" (UID: \"62f4330c-55ea-46a3-9027-f1723405db8e\") " pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" Jul 17 21:23:35.895651 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.895508 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62f4330c-55ea-46a3-9027-f1723405db8e-tmp\") pod \"kube-auth-proxy-5fbb8dc498-xsnxw\" (UID: \"62f4330c-55ea-46a3-9027-f1723405db8e\") " pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" Jul 17 21:23:35.895651 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.895583 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f4330c-55ea-46a3-9027-f1723405db8e-tls-certs\") pod \"kube-auth-proxy-5fbb8dc498-xsnxw\" (UID: \"62f4330c-55ea-46a3-9027-f1723405db8e\") " pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" Jul 17 21:23:35.945995 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.945951 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:35.996700 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.996618 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62f4330c-55ea-46a3-9027-f1723405db8e-tmp\") pod \"kube-auth-proxy-5fbb8dc498-xsnxw\" (UID: \"62f4330c-55ea-46a3-9027-f1723405db8e\") " pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" Jul 17 21:23:35.996700 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.996671 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f4330c-55ea-46a3-9027-f1723405db8e-tls-certs\") pod \"kube-auth-proxy-5fbb8dc498-xsnxw\" (UID: \"62f4330c-55ea-46a3-9027-f1723405db8e\") " pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" Jul 17 21:23:35.996861 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.996722 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6776k\" (UniqueName: \"kubernetes.io/projected/62f4330c-55ea-46a3-9027-f1723405db8e-kube-api-access-6776k\") pod \"kube-auth-proxy-5fbb8dc498-xsnxw\" (UID: \"62f4330c-55ea-46a3-9027-f1723405db8e\") " pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" Jul 17 21:23:35.999009 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.998984 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62f4330c-55ea-46a3-9027-f1723405db8e-tmp\") pod \"kube-auth-proxy-5fbb8dc498-xsnxw\" (UID: \"62f4330c-55ea-46a3-9027-f1723405db8e\") " pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" Jul 17 21:23:35.999124 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:35.999107 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f4330c-55ea-46a3-9027-f1723405db8e-tls-certs\") pod \"kube-auth-proxy-5fbb8dc498-xsnxw\" (UID: \"62f4330c-55ea-46a3-9027-f1723405db8e\") " pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" Jul 17 21:23:36.017972 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.017949 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6776k\" (UniqueName: \"kubernetes.io/projected/62f4330c-55ea-46a3-9027-f1723405db8e-kube-api-access-6776k\") pod \"kube-auth-proxy-5fbb8dc498-xsnxw\" (UID: \"62f4330c-55ea-46a3-9027-f1723405db8e\") " pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" Jul 17 21:23:36.114258 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.114231 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c7kp5"] Jul 17 21:23:36.116121 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.116106 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" Jul 17 21:23:36.118881 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.118864 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" Jul 17 21:23:36.118957 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.118937 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 17 21:23:36.119170 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.119156 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2xjmh\"" Jul 17 21:23:36.152033 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.152000 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c7kp5"] Jul 17 21:23:36.197729 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.197700 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvrms\" (UniqueName: \"kubernetes.io/projected/daabaacb-517b-4583-b831-4ee73d8dc255-kube-api-access-kvrms\") pod \"odh-model-controller-7998c67dc4-c7kp5\" (UID: \"daabaacb-517b-4583-b831-4ee73d8dc255\") " pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" Jul 17 21:23:36.197903 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.197882 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/daabaacb-517b-4583-b831-4ee73d8dc255-cert\") pod \"odh-model-controller-7998c67dc4-c7kp5\" (UID: \"daabaacb-517b-4583-b831-4ee73d8dc255\") " pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" Jul 17 21:23:36.227432 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.227333 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:36.273534 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.273506 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw"] Jul 17 21:23:36.276867 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:23:36.276835 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f4330c_55ea_46a3_9027_f1723405db8e.slice/crio-4c9b40e5811c4c823ca7cbdf13b73c735770fe34d44d455326499e3e7a289cd8 WatchSource:0}: Error finding container 4c9b40e5811c4c823ca7cbdf13b73c735770fe34d44d455326499e3e7a289cd8: Status 404 returned error can't find the container with id 4c9b40e5811c4c823ca7cbdf13b73c735770fe34d44d455326499e3e7a289cd8 Jul 17 21:23:36.299201 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.299178 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/daabaacb-517b-4583-b831-4ee73d8dc255-cert\") pod \"odh-model-controller-7998c67dc4-c7kp5\" (UID: \"daabaacb-517b-4583-b831-4ee73d8dc255\") " pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" Jul 17 21:23:36.299274 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.299211 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvrms\" (UniqueName: \"kubernetes.io/projected/daabaacb-517b-4583-b831-4ee73d8dc255-kube-api-access-kvrms\") pod \"odh-model-controller-7998c67dc4-c7kp5\" (UID: \"daabaacb-517b-4583-b831-4ee73d8dc255\") " pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" Jul 17 21:23:36.299317 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:23:36.299308 2554 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 17 21:23:36.299368 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:23:36.299358 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daabaacb-517b-4583-b831-4ee73d8dc255-cert podName:daabaacb-517b-4583-b831-4ee73d8dc255 nodeName:}" failed. No retries permitted until 2026-07-17 21:23:36.799345166 +0000 UTC m=+397.568693122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/daabaacb-517b-4583-b831-4ee73d8dc255-cert") pod "odh-model-controller-7998c67dc4-c7kp5" (UID: "daabaacb-517b-4583-b831-4ee73d8dc255") : secret "odh-model-controller-webhook-cert" not found Jul 17 21:23:36.311349 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.311326 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvrms\" (UniqueName: \"kubernetes.io/projected/daabaacb-517b-4583-b831-4ee73d8dc255-kube-api-access-kvrms\") pod \"odh-model-controller-7998c67dc4-c7kp5\" (UID: \"daabaacb-517b-4583-b831-4ee73d8dc255\") " pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" Jul 17 21:23:36.322335 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.322310 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" event={"ID":"62f4330c-55ea-46a3-9027-f1723405db8e","Type":"ContainerStarted","Data":"4c9b40e5811c4c823ca7cbdf13b73c735770fe34d44d455326499e3e7a289cd8"} Jul 17 21:23:36.344704 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.344675 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:36.450852 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.450819 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:36.542838 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.542729 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:36.804109 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.804028 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/daabaacb-517b-4583-b831-4ee73d8dc255-cert\") pod \"odh-model-controller-7998c67dc4-c7kp5\" (UID: \"daabaacb-517b-4583-b831-4ee73d8dc255\") " pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" Jul 17 21:23:36.806494 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:36.806466 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/daabaacb-517b-4583-b831-4ee73d8dc255-cert\") pod \"odh-model-controller-7998c67dc4-c7kp5\" (UID: \"daabaacb-517b-4583-b831-4ee73d8dc255\") " pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" Jul 17 21:23:37.027159 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.027107 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" Jul 17 21:23:37.188701 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.188668 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c7kp5"] Jul 17 21:23:37.192154 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:23:37.192117 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaabaacb_517b_4583_b831_4ee73d8dc255.slice/crio-aadac083ab41d3b40f8ba44696193de119e4ca5efd32bdaa3b2e38d63bb43895 WatchSource:0}: Error finding container aadac083ab41d3b40f8ba44696193de119e4ca5efd32bdaa3b2e38d63bb43895: Status 404 returned error can't find the container with id aadac083ab41d3b40f8ba44696193de119e4ca5efd32bdaa3b2e38d63bb43895 Jul 17 21:23:37.196092 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.196028 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:37.246383 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.246341 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dvw4f"] Jul 17 21:23:37.248850 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.248831 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" Jul 17 21:23:37.251494 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.251472 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 17 21:23:37.251770 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.251753 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-jdlrd\"" Jul 17 21:23:37.259197 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.259174 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dvw4f"] Jul 17 21:23:37.308268 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.308241 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/410071d2-6321-4391-9165-39499b01b2d8-tls-certs\") pod \"model-serving-api-54d89bd5fc-dvw4f\" (UID: \"410071d2-6321-4391-9165-39499b01b2d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" Jul 17 21:23:37.308407 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.308272 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sd2f\" (UniqueName: \"kubernetes.io/projected/410071d2-6321-4391-9165-39499b01b2d8-kube-api-access-5sd2f\") pod \"model-serving-api-54d89bd5fc-dvw4f\" (UID: \"410071d2-6321-4391-9165-39499b01b2d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" Jul 17 21:23:37.329982 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.329933 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" event={"ID":"daabaacb-517b-4583-b831-4ee73d8dc255","Type":"ContainerStarted","Data":"aadac083ab41d3b40f8ba44696193de119e4ca5efd32bdaa3b2e38d63bb43895"} Jul 17 21:23:37.409381 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.409289 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/410071d2-6321-4391-9165-39499b01b2d8-tls-certs\") pod \"model-serving-api-54d89bd5fc-dvw4f\" (UID: \"410071d2-6321-4391-9165-39499b01b2d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" Jul 17 21:23:37.409381 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.409342 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sd2f\" (UniqueName: \"kubernetes.io/projected/410071d2-6321-4391-9165-39499b01b2d8-kube-api-access-5sd2f\") pod \"model-serving-api-54d89bd5fc-dvw4f\" (UID: \"410071d2-6321-4391-9165-39499b01b2d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" Jul 17 21:23:37.409611 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:23:37.409533 2554 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 17 21:23:37.409658 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:23:37.409630 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/410071d2-6321-4391-9165-39499b01b2d8-tls-certs podName:410071d2-6321-4391-9165-39499b01b2d8 nodeName:}" failed. No retries permitted until 2026-07-17 21:23:37.909608368 +0000 UTC m=+398.678956331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/410071d2-6321-4391-9165-39499b01b2d8-tls-certs") pod "model-serving-api-54d89bd5fc-dvw4f" (UID: "410071d2-6321-4391-9165-39499b01b2d8") : secret "model-serving-api-tls" not found Jul 17 21:23:37.425684 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.425659 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sd2f\" (UniqueName: \"kubernetes.io/projected/410071d2-6321-4391-9165-39499b01b2d8-kube-api-access-5sd2f\") pod \"model-serving-api-54d89bd5fc-dvw4f\" (UID: \"410071d2-6321-4391-9165-39499b01b2d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" Jul 17 21:23:37.471066 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.471035 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk96w"] Jul 17 21:23:37.473285 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.473263 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.475511 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.475482 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:37.475971 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.475952 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 17 21:23:37.475971 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.475968 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 17 21:23:37.475971 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.475960 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 17 21:23:37.475971 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.476114 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 17 21:23:37.476738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.476145 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-747hd\"" Jul 17 21:23:37.487894 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.487873 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk96w"] Jul 17 21:23:37.574514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.574479 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:37.611494 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.611462 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-token\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.611689 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.611499 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.611689 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.611527 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8xdq\" (UniqueName: \"kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-kube-api-access-d8xdq\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.611689 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.611606 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.611689 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.611664 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c624f6d2-829b-4289-9743-9c8f75f09da6-local-certs\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.611908 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.611691 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.611908 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.611743 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-cacerts\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.712888 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.712812 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-cacerts\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.713058 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.712911 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-token\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.713058 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.712946 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.713058 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.712974 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xdq\" (UniqueName: \"kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-kube-api-access-d8xdq\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.713058 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.713006 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.713058 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.713056 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c624f6d2-829b-4289-9743-9c8f75f09da6-local-certs\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.713309 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.713080 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.713953 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.713727 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.715741 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.715713 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-cacerts\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.715843 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.715766 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c624f6d2-829b-4289-9743-9c8f75f09da6-local-certs\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.715843 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.715768 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.715843 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.715833 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.723309 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.723289 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-token\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.723453 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.723436 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8xdq\" (UniqueName: \"kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-kube-api-access-d8xdq\") pod \"istiod-openshift-gateway-7574459656-lk96w\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.784375 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.784349 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:37.915446 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.915412 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/410071d2-6321-4391-9165-39499b01b2d8-tls-certs\") pod \"model-serving-api-54d89bd5fc-dvw4f\" (UID: \"410071d2-6321-4391-9165-39499b01b2d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" Jul 17 21:23:37.918395 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.918367 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/410071d2-6321-4391-9165-39499b01b2d8-tls-certs\") pod \"model-serving-api-54d89bd5fc-dvw4f\" (UID: \"410071d2-6321-4391-9165-39499b01b2d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" Jul 17 21:23:37.950803 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:37.950765 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk96w"] Jul 17 21:23:37.950966 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:23:37.950940 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc624f6d2_829b_4289_9743_9c8f75f09da6.slice/crio-c5881ea6411f7172c84746cd82eac65cf04481cf2fe84d99bb4c1bb0013195f8 WatchSource:0}: Error finding container c5881ea6411f7172c84746cd82eac65cf04481cf2fe84d99bb4c1bb0013195f8: Status 404 returned error can't find the container with id c5881ea6411f7172c84746cd82eac65cf04481cf2fe84d99bb4c1bb0013195f8 Jul 17 21:23:38.162584 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:38.162522 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" Jul 17 21:23:38.340240 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:38.340053 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" event={"ID":"c624f6d2-829b-4289-9743-9c8f75f09da6","Type":"ContainerStarted","Data":"c5881ea6411f7172c84746cd82eac65cf04481cf2fe84d99bb4c1bb0013195f8"} Jul 17 21:23:38.340240 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:38.340132 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dvw4f"] Jul 17 21:23:38.696354 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:38.696318 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:39.082352 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:23:39.082319 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod410071d2_6321_4391_9165_39499b01b2d8.slice/crio-e2d34fdf67232cf1679a6acab8670a1e1f7d5701a3593699766bc327fd659ac3 WatchSource:0}: Error finding container e2d34fdf67232cf1679a6acab8670a1e1f7d5701a3593699766bc327fd659ac3: Status 404 returned error can't find the container with id e2d34fdf67232cf1679a6acab8670a1e1f7d5701a3593699766bc327fd659ac3 Jul 17 21:23:39.347113 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:39.347012 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" event={"ID":"410071d2-6321-4391-9165-39499b01b2d8","Type":"ContainerStarted","Data":"e2d34fdf67232cf1679a6acab8670a1e1f7d5701a3593699766bc327fd659ac3"} Jul 17 21:23:41.252989 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:41.252957 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:41.352674 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:41.352636 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:41.959038 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:41.959007 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 17 21:23:41.959147 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:41.959065 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 17 21:23:42.363608 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.363508 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" event={"ID":"410071d2-6321-4391-9165-39499b01b2d8","Type":"ContainerStarted","Data":"5548ce331f45eaf1db94d924469f7192fc762c1ac7a3a4552d71258fc7826aeb"} Jul 17 21:23:42.364074 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.363633 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" Jul 17 21:23:42.365098 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.365072 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" event={"ID":"62f4330c-55ea-46a3-9027-f1723405db8e","Type":"ContainerStarted","Data":"bf72e15a3f01aa72cf55861740b2ed8414b086504447e65f2418c437e7d8bae8"} Jul 17 21:23:42.366519 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.366495 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" event={"ID":"daabaacb-517b-4583-b831-4ee73d8dc255","Type":"ContainerStarted","Data":"d5f216c56357f31840ba280d762d75d19d5d71fec5f9de4decae9665cb9a6ce3"} Jul 17 21:23:42.366631 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.366582 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" Jul 17 21:23:42.368058 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.368041 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" event={"ID":"c624f6d2-829b-4289-9743-9c8f75f09da6","Type":"ContainerStarted","Data":"ecbcbc4f4f3e2c4ce3c6c073637492b17078c02f973ec0abb084e1fb262382cb"} Jul 17 21:23:42.368164 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.368153 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:42.382179 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.382141 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" podStartSLOduration=2.5077000529999998 podStartE2EDuration="5.382128842s" totalImagesPullingTime="2.874428789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:23:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:23:42.380606675 +0000 UTC m=+403.149954653" watchObservedRunningTime="2026-07-17 21:23:42.382128842 +0000 UTC m=+403.151476820" Jul 17 21:23:42.401890 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.401842 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" podStartSLOduration=1.645560532 podStartE2EDuration="6.401821417s" totalImagesPullingTime="4.756260885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:23:42.399412264 +0000 UTC m=+403.168760245" watchObservedRunningTime="2026-07-17 21:23:42.401821417 +0000 UTC m=+403.171169397" Jul 17 21:23:42.422051 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.422003 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5fbb8dc498-xsnxw" podStartSLOduration=2.141646164 podStartE2EDuration="7.421988236s" totalImagesPullingTime="5.280342072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:23:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:23:42.421116833 +0000 UTC m=+403.190464811" watchObservedRunningTime="2026-07-17 21:23:42.421988236 +0000 UTC m=+403.191336213" Jul 17 21:23:42.450966 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.450916 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" podStartSLOduration=1.445646363 podStartE2EDuration="5.450896778s" totalImagesPullingTime="4.005250415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:23:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:23:42.448272573 +0000 UTC m=+403.217620552" watchObservedRunningTime="2026-07-17 21:23:42.450896778 +0000 UTC m=+403.220244756" Jul 17 21:23:42.886409 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.886368 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:42.985194 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:42.985140 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:43.170065 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.169969 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:43.193531 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.193504 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x"] Jul 17 21:23:43.196814 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.196793 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.203864 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.203844 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ddgxb\"" Jul 17 21:23:43.218830 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.218806 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x"] Jul 17 21:23:43.263789 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.263766 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.263916 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.263810 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.263916 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.263828 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.263991 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.263918 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.263991 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.263954 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a89ca970-488e-40e2-b7ad-660b941a47c5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.263991 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.263981 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w6fc\" (UniqueName: \"kubernetes.io/projected/a89ca970-488e-40e2-b7ad-660b941a47c5-kube-api-access-7w6fc\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.264086 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.264025 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.264086 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.264053 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.264156 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.264097 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.364856 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.364826 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.364856 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.364858 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a89ca970-488e-40e2-b7ad-660b941a47c5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.364879 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w6fc\" (UniqueName: \"kubernetes.io/projected/a89ca970-488e-40e2-b7ad-660b941a47c5-kube-api-access-7w6fc\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.365006 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.365059 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.365094 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.365144 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.365194 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.365223 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.365250 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365638 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.365397 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365638 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.365478 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365638 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.365570 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.365638 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.365636 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a89ca970-488e-40e2-b7ad-660b941a47c5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.367505 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.367488 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.367729 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.367713 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.373939 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.373911 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:43.381309 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.381291 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a89ca970-488e-40e2-b7ad-660b941a47c5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.385827 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.385804 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w6fc\" (UniqueName: \"kubernetes.io/projected/a89ca970-488e-40e2-b7ad-660b941a47c5-kube-api-access-7w6fc\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qq65x\" (UID: \"a89ca970-488e-40e2-b7ad-660b941a47c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.413556 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.413517 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:43.507209 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.507135 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:43.518835 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.518804 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:43.652043 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:43.651978 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x"] Jul 17 21:23:43.654173 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:23:43.654145 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89ca970_488e_40e2_b7ad_660b941a47c5.slice/crio-a8246d2cf6eb190a31435d930d2cedb0b0255d410201cb09035146970666132e WatchSource:0}: Error finding container a8246d2cf6eb190a31435d930d2cedb0b0255d410201cb09035146970666132e: Status 404 returned error can't find the container with id a8246d2cf6eb190a31435d930d2cedb0b0255d410201cb09035146970666132e Jul 17 21:23:44.379448 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:44.379409 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" event={"ID":"a89ca970-488e-40e2-b7ad-660b941a47c5","Type":"ContainerStarted","Data":"a8246d2cf6eb190a31435d930d2cedb0b0255d410201cb09035146970666132e"} Jul 17 21:23:46.230261 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:46.230229 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 17 21:23:46.230665 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:46.230299 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 17 21:23:46.230665 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:46.230326 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 17 21:23:46.391646 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:46.391585 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" event={"ID":"a89ca970-488e-40e2-b7ad-660b941a47c5","Type":"ContainerStarted","Data":"3e65ada38c2ffaa3fc44c29169b1c01dd5e6c0dc1de7c2fb80089c33fdf445d2"} Jul 17 21:23:46.437323 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:46.437273 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" podStartSLOduration=0.86337704 podStartE2EDuration="3.437256968s" totalImagesPullingTime="2.573879928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:23:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:23:46.435337234 +0000 UTC m=+407.204685212" watchObservedRunningTime="2026-07-17 21:23:46.437256968 +0000 UTC m=+407.206605019" Jul 17 21:23:46.507874 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:46.507797 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:47.248788 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.248745 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:47.449869 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.449829 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:47.476256 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.476223 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s"] Jul 17 21:23:47.479754 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.479736 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" Jul 17 21:23:47.482749 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.482727 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd"] Jul 17 21:23:47.484526 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.484509 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 17 21:23:47.485257 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.485235 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-svc9z\"" Jul 17 21:23:47.486075 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.486059 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" Jul 17 21:23:47.488781 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.488755 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-b77vt\"" Jul 17 21:23:47.489138 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.489118 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 17 21:23:47.511533 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.511469 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s"] Jul 17 21:23:47.514322 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.514296 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd"] Jul 17 21:23:47.518840 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.518820 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:47.602963 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.602937 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sp9h\" (UniqueName: \"kubernetes.io/projected/0f19398a-d921-4131-85d9-a0245a13d4c2-kube-api-access-2sp9h\") pod \"llmisvc-controller-manager-777c6bd7d5-dzl9s\" (UID: \"0f19398a-d921-4131-85d9-a0245a13d4c2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" Jul 17 21:23:47.603123 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.602988 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f19398a-d921-4131-85d9-a0245a13d4c2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dzl9s\" (UID: \"0f19398a-d921-4131-85d9-a0245a13d4c2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" Jul 17 21:23:47.603123 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.603005 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kr7t\" (UniqueName: \"kubernetes.io/projected/b5b821f5-5bd8-423d-b0c7-bbcebc1832d9-kube-api-access-7kr7t\") pod \"kserve-controller-manager-d6f4b65b8-xnlqd\" (UID: \"b5b821f5-5bd8-423d-b0c7-bbcebc1832d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" Jul 17 21:23:47.603123 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.603085 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5b821f5-5bd8-423d-b0c7-bbcebc1832d9-cert\") pod \"kserve-controller-manager-d6f4b65b8-xnlqd\" (UID: \"b5b821f5-5bd8-423d-b0c7-bbcebc1832d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" Jul 17 21:23:47.703512 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.703481 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f19398a-d921-4131-85d9-a0245a13d4c2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dzl9s\" (UID: \"0f19398a-d921-4131-85d9-a0245a13d4c2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" Jul 17 21:23:47.703512 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.703512 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kr7t\" (UniqueName: \"kubernetes.io/projected/b5b821f5-5bd8-423d-b0c7-bbcebc1832d9-kube-api-access-7kr7t\") pod \"kserve-controller-manager-d6f4b65b8-xnlqd\" (UID: \"b5b821f5-5bd8-423d-b0c7-bbcebc1832d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" Jul 17 21:23:47.703705 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.703564 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5b821f5-5bd8-423d-b0c7-bbcebc1832d9-cert\") pod \"kserve-controller-manager-d6f4b65b8-xnlqd\" (UID: \"b5b821f5-5bd8-423d-b0c7-bbcebc1832d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" Jul 17 21:23:47.703705 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:23:47.703649 2554 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 17 21:23:47.703705 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:23:47.703659 2554 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 17 21:23:47.703705 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:23:47.703692 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5b821f5-5bd8-423d-b0c7-bbcebc1832d9-cert podName:b5b821f5-5bd8-423d-b0c7-bbcebc1832d9 nodeName:}" failed. No retries permitted until 2026-07-17 21:23:48.203677627 +0000 UTC m=+408.973025582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b5b821f5-5bd8-423d-b0c7-bbcebc1832d9-cert") pod "kserve-controller-manager-d6f4b65b8-xnlqd" (UID: "b5b821f5-5bd8-423d-b0c7-bbcebc1832d9") : secret "kserve-webhook-server-cert" not found Jul 17 21:23:47.703869 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.703706 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sp9h\" (UniqueName: \"kubernetes.io/projected/0f19398a-d921-4131-85d9-a0245a13d4c2-kube-api-access-2sp9h\") pod \"llmisvc-controller-manager-777c6bd7d5-dzl9s\" (UID: \"0f19398a-d921-4131-85d9-a0245a13d4c2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" Jul 17 21:23:47.703869 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:23:47.703723 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f19398a-d921-4131-85d9-a0245a13d4c2-cert podName:0f19398a-d921-4131-85d9-a0245a13d4c2 nodeName:}" failed. No retries permitted until 2026-07-17 21:23:48.203705062 +0000 UTC m=+408.973053042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0f19398a-d921-4131-85d9-a0245a13d4c2-cert") pod "llmisvc-controller-manager-777c6bd7d5-dzl9s" (UID: "0f19398a-d921-4131-85d9-a0245a13d4c2") : secret "llmisvc-webhook-server-cert" not found Jul 17 21:23:47.730311 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.730282 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sp9h\" (UniqueName: \"kubernetes.io/projected/0f19398a-d921-4131-85d9-a0245a13d4c2-kube-api-access-2sp9h\") pod \"llmisvc-controller-manager-777c6bd7d5-dzl9s\" (UID: \"0f19398a-d921-4131-85d9-a0245a13d4c2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" Jul 17 21:23:47.742128 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.742093 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:47.743236 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.743219 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kr7t\" (UniqueName: \"kubernetes.io/projected/b5b821f5-5bd8-423d-b0c7-bbcebc1832d9-kube-api-access-7kr7t\") pod \"kserve-controller-manager-d6f4b65b8-xnlqd\" (UID: \"b5b821f5-5bd8-423d-b0c7-bbcebc1832d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" Jul 17 21:23:47.997012 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:47.996980 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:48.190590 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.190535 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:48.207669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.207639 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f19398a-d921-4131-85d9-a0245a13d4c2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dzl9s\" (UID: \"0f19398a-d921-4131-85d9-a0245a13d4c2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" Jul 17 21:23:48.207821 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.207694 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5b821f5-5bd8-423d-b0c7-bbcebc1832d9-cert\") pod \"kserve-controller-manager-d6f4b65b8-xnlqd\" (UID: \"b5b821f5-5bd8-423d-b0c7-bbcebc1832d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" Jul 17 21:23:48.210250 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.210228 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f19398a-d921-4131-85d9-a0245a13d4c2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dzl9s\" (UID: \"0f19398a-d921-4131-85d9-a0245a13d4c2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" Jul 17 21:23:48.210356 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.210266 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5b821f5-5bd8-423d-b0c7-bbcebc1832d9-cert\") pod \"kserve-controller-manager-d6f4b65b8-xnlqd\" (UID: \"b5b821f5-5bd8-423d-b0c7-bbcebc1832d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" Jul 17 21:23:48.323915 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.323884 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:48.391420 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.391394 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" Jul 17 21:23:48.400931 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.400906 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:48.401917 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.401895 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qq65x" Jul 17 21:23:48.402029 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.401900 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" Jul 17 21:23:48.567741 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.567715 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s"] Jul 17 21:23:48.569426 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:23:48.569395 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0f19398a_d921_4131_85d9_a0245a13d4c2.slice/crio-a5c8aec142eb71a6e61e49e1b881d05753b96e2605c34e7092bfc829816ed482 WatchSource:0}: Error finding container a5c8aec142eb71a6e61e49e1b881d05753b96e2605c34e7092bfc829816ed482: Status 404 returned error can't find the container with id a5c8aec142eb71a6e61e49e1b881d05753b96e2605c34e7092bfc829816ed482 Jul 17 21:23:48.608409 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:48.608385 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd"] Jul 17 21:23:48.610865 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:23:48.610840 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5b821f5_5bd8_423d_b0c7_bbcebc1832d9.slice/crio-84014fd7cf44536fe7f54d8269d13aaa9b635054db69abec6acce7aae3714841 WatchSource:0}: Error finding container 84014fd7cf44536fe7f54d8269d13aaa9b635054db69abec6acce7aae3714841: Status 404 returned error can't find the container with id 84014fd7cf44536fe7f54d8269d13aaa9b635054db69abec6acce7aae3714841 Jul 17 21:23:49.219371 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.219333 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk96w"] Jul 17 21:23:49.219635 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.219607 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" podUID="c624f6d2-829b-4289-9743-9c8f75f09da6" containerName="discovery" containerID="cri-o://ecbcbc4f4f3e2c4ce3c6c073637492b17078c02f973ec0abb084e1fb262382cb" gracePeriod=30 Jul 17 21:23:49.375109 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.374927 2554 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-lk96w container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.25:8080/ready\": dial tcp 10.132.0.25:8080: connect: connection refused" start-of-body= Jul 17 21:23:49.375109 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.374982 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" podUID="c624f6d2-829b-4289-9743-9c8f75f09da6" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.25:8080/ready\": dial tcp 10.132.0.25:8080: connect: connection refused" Jul 17 21:23:49.406296 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.406267 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" event={"ID":"b5b821f5-5bd8-423d-b0c7-bbcebc1832d9","Type":"ContainerStarted","Data":"84014fd7cf44536fe7f54d8269d13aaa9b635054db69abec6acce7aae3714841"} Jul 17 21:23:49.407815 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.407793 2554 generic.go:356] "Generic (PLEG): container finished" podID="c624f6d2-829b-4289-9743-9c8f75f09da6" containerID="ecbcbc4f4f3e2c4ce3c6c073637492b17078c02f973ec0abb084e1fb262382cb" exitCode=0 Jul 17 21:23:49.407930 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.407861 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" event={"ID":"c624f6d2-829b-4289-9743-9c8f75f09da6","Type":"ContainerDied","Data":"ecbcbc4f4f3e2c4ce3c6c073637492b17078c02f973ec0abb084e1fb262382cb"} Jul 17 21:23:49.408950 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.408929 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" event={"ID":"0f19398a-d921-4131-85d9-a0245a13d4c2","Type":"ContainerStarted","Data":"a5c8aec142eb71a6e61e49e1b881d05753b96e2605c34e7092bfc829816ed482"} Jul 17 21:23:49.542292 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.542266 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:49.618714 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.618684 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-cacerts\") pod \"c624f6d2-829b-4289-9743-9c8f75f09da6\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " Jul 17 21:23:49.618880 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.618741 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-ca-configmap\") pod \"c624f6d2-829b-4289-9743-9c8f75f09da6\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " Jul 17 21:23:49.618880 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.618771 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-token\") pod \"c624f6d2-829b-4289-9743-9c8f75f09da6\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " Jul 17 21:23:49.618880 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.618792 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-kubeconfig\") pod \"c624f6d2-829b-4289-9743-9c8f75f09da6\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " Jul 17 21:23:49.618880 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.618837 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c624f6d2-829b-4289-9743-9c8f75f09da6-local-certs\") pod \"c624f6d2-829b-4289-9743-9c8f75f09da6\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " Jul 17 21:23:49.619083 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.618885 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-dns-cert\") pod \"c624f6d2-829b-4289-9743-9c8f75f09da6\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " Jul 17 21:23:49.619083 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.618926 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8xdq\" (UniqueName: \"kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-kube-api-access-d8xdq\") pod \"c624f6d2-829b-4289-9743-9c8f75f09da6\" (UID: \"c624f6d2-829b-4289-9743-9c8f75f09da6\") " Jul 17 21:23:49.619969 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.619934 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "c624f6d2-829b-4289-9743-9c8f75f09da6" (UID: "c624f6d2-829b-4289-9743-9c8f75f09da6"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 21:23:49.622168 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.622129 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-token" (OuterVolumeSpecName: "istio-token") pod "c624f6d2-829b-4289-9743-9c8f75f09da6" (UID: "c624f6d2-829b-4289-9743-9c8f75f09da6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 21:23:49.622355 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.622320 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-kube-api-access-d8xdq" (OuterVolumeSpecName: "kube-api-access-d8xdq") pod "c624f6d2-829b-4289-9743-9c8f75f09da6" (UID: "c624f6d2-829b-4289-9743-9c8f75f09da6"). InnerVolumeSpecName "kube-api-access-d8xdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 21:23:49.622355 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.622338 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c624f6d2-829b-4289-9743-9c8f75f09da6-local-certs" (OuterVolumeSpecName: "local-certs") pod "c624f6d2-829b-4289-9743-9c8f75f09da6" (UID: "c624f6d2-829b-4289-9743-9c8f75f09da6"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 21:23:49.623090 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.623051 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "c624f6d2-829b-4289-9743-9c8f75f09da6" (UID: "c624f6d2-829b-4289-9743-9c8f75f09da6"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 21:23:49.623524 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.623500 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-cacerts" (OuterVolumeSpecName: "cacerts") pod "c624f6d2-829b-4289-9743-9c8f75f09da6" (UID: "c624f6d2-829b-4289-9743-9c8f75f09da6"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 21:23:49.623876 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.623853 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "c624f6d2-829b-4289-9743-9c8f75f09da6" (UID: "c624f6d2-829b-4289-9743-9c8f75f09da6"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 21:23:49.629718 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.629637 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:49.721585 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.721361 2554 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c624f6d2-829b-4289-9743-9c8f75f09da6-local-certs\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:23:49.721585 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.721394 2554 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-dns-cert\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:23:49.721585 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.721416 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8xdq\" (UniqueName: \"kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-kube-api-access-d8xdq\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:23:49.721585 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.721430 2554 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-cacerts\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:23:49.721585 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.721445 2554 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-csr-ca-configmap\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:23:49.721585 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.721460 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-token\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:23:49.721585 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.721478 2554 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c624f6d2-829b-4289-9743-9c8f75f09da6-istio-kubeconfig\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:23:49.798563 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:49.798452 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:50.416049 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:50.416012 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" Jul 17 21:23:50.416528 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:50.416051 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk96w" event={"ID":"c624f6d2-829b-4289-9743-9c8f75f09da6","Type":"ContainerDied","Data":"c5881ea6411f7172c84746cd82eac65cf04481cf2fe84d99bb4c1bb0013195f8"} Jul 17 21:23:50.416528 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:50.416098 2554 scope.go:117] "RemoveContainer" containerID="ecbcbc4f4f3e2c4ce3c6c073637492b17078c02f973ec0abb084e1fb262382cb" Jul 17 21:23:50.447209 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:50.447183 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk96w"] Jul 17 21:23:50.468463 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:50.468432 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk96w"] Jul 17 21:23:51.721281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:51.721244 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c624f6d2-829b-4289-9743-9c8f75f09da6" path="/var/lib/kubelet/pods/c624f6d2-829b-4289-9743-9c8f75f09da6/volumes" Jul 17 21:23:52.426732 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:52.426693 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" event={"ID":"0f19398a-d921-4131-85d9-a0245a13d4c2","Type":"ContainerStarted","Data":"6e793518ddbeb803a2d9fb8624e0594630b924c425644de0264e51680e08af22"} Jul 17 21:23:52.426907 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:52.426789 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" Jul 17 21:23:52.428211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:52.428186 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" event={"ID":"b5b821f5-5bd8-423d-b0c7-bbcebc1832d9","Type":"ContainerStarted","Data":"1d9c6f38517a61aa11498f7f8cc5ec487def700531cfd688883c78a751003c44"} Jul 17 21:23:52.428360 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:52.428238 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" Jul 17 21:23:52.444757 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:52.444711 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" podStartSLOduration=1.9992320829999999 podStartE2EDuration="5.44469691s" totalImagesPullingTime="3.445464827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:23:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:23:52.443789657 +0000 UTC m=+413.213137635" watchObservedRunningTime="2026-07-17 21:23:52.44469691 +0000 UTC m=+413.214044884" Jul 17 21:23:52.466212 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:52.466159 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" podStartSLOduration=2.065390855 podStartE2EDuration="5.466144952s" totalImagesPullingTime="3.400754097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:23:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:23:52.465772949 +0000 UTC m=+413.235120951" watchObservedRunningTime="2026-07-17 21:23:52.466144952 +0000 UTC m=+413.235492932" Jul 17 21:23:53.374921 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:53.374892 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-c7kp5" Jul 17 21:23:53.376599 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:53.376580 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-dvw4f" Jul 17 21:23:58.272658 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:58.272605 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:23:58.373378 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:23:58.373317 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:24:23.435009 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:23.434929 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dzl9s" Jul 17 21:24:23.437867 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:23.437851 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xnlqd" Jul 17 21:24:28.621094 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:28.621060 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:24:28.691594 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:28.691568 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:24:36.471812 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.471769 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:24:36.571571 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.571520 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:24:36.595087 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.595057 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d2gd2"] Jul 17 21:24:36.595451 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.595434 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c624f6d2-829b-4289-9743-9c8f75f09da6" containerName="discovery" Jul 17 21:24:36.595535 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.595454 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c624f6d2-829b-4289-9743-9c8f75f09da6" containerName="discovery" Jul 17 21:24:36.595614 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.595563 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c624f6d2-829b-4289-9743-9c8f75f09da6" containerName="discovery" Jul 17 21:24:36.597243 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.597222 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" Jul 17 21:24:36.600007 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.599986 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 17 21:24:36.600126 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.600036 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 17 21:24:36.600126 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.600087 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-5s65w\"" Jul 17 21:24:36.609085 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.609062 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d2gd2"] Jul 17 21:24:36.641847 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.641822 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:24:36.697861 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.697835 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp82n\" (UniqueName: \"kubernetes.io/projected/0cec39d3-dd08-4376-a02c-a1d2e18c1b5d-kube-api-access-qp82n\") pod \"kuadrant-operator-catalog-d2gd2\" (UID: \"0cec39d3-dd08-4376-a02c-a1d2e18c1b5d\") " pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" Jul 17 21:24:36.798287 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.798261 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp82n\" (UniqueName: \"kubernetes.io/projected/0cec39d3-dd08-4376-a02c-a1d2e18c1b5d-kube-api-access-qp82n\") pod \"kuadrant-operator-catalog-d2gd2\" (UID: \"0cec39d3-dd08-4376-a02c-a1d2e18c1b5d\") " pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" Jul 17 21:24:36.808452 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.808433 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp82n\" (UniqueName: \"kubernetes.io/projected/0cec39d3-dd08-4376-a02c-a1d2e18c1b5d-kube-api-access-qp82n\") pod \"kuadrant-operator-catalog-d2gd2\" (UID: \"0cec39d3-dd08-4376-a02c-a1d2e18c1b5d\") " pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" Jul 17 21:24:36.908676 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.908649 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" Jul 17 21:24:36.947593 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:36.947521 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d2gd2"] Jul 17 21:24:37.037314 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:37.037284 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d2gd2"] Jul 17 21:24:37.040391 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:24:37.040367 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cec39d3_dd08_4376_a02c_a1d2e18c1b5d.slice/crio-3e80fed384a848c081db8cf8808b00a6ffaaf7eddc7313e5ab1e1ee8bc339668 WatchSource:0}: Error finding container 3e80fed384a848c081db8cf8808b00a6ffaaf7eddc7313e5ab1e1ee8bc339668: Status 404 returned error can't find the container with id 3e80fed384a848c081db8cf8808b00a6ffaaf7eddc7313e5ab1e1ee8bc339668 Jul 17 21:24:37.162595 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:37.162489 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t2z5f"] Jul 17 21:24:37.165430 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:37.165411 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" Jul 17 21:24:37.174070 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:37.174048 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t2z5f"] Jul 17 21:24:37.302677 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:37.302639 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgbw4\" (UniqueName: \"kubernetes.io/projected/5ba6a147-13f4-467b-966d-faeb5482199f-kube-api-access-wgbw4\") pod \"kuadrant-operator-catalog-t2z5f\" (UID: \"5ba6a147-13f4-467b-966d-faeb5482199f\") " pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" Jul 17 21:24:37.403444 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:37.403406 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgbw4\" (UniqueName: \"kubernetes.io/projected/5ba6a147-13f4-467b-966d-faeb5482199f-kube-api-access-wgbw4\") pod \"kuadrant-operator-catalog-t2z5f\" (UID: \"5ba6a147-13f4-467b-966d-faeb5482199f\") " pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" Jul 17 21:24:37.415627 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:37.415539 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgbw4\" (UniqueName: \"kubernetes.io/projected/5ba6a147-13f4-467b-966d-faeb5482199f-kube-api-access-wgbw4\") pod \"kuadrant-operator-catalog-t2z5f\" (UID: \"5ba6a147-13f4-467b-966d-faeb5482199f\") " pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" Jul 17 21:24:37.475769 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:37.475740 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" Jul 17 21:24:37.611100 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:37.611076 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t2z5f"] Jul 17 21:24:37.613359 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:24:37.613326 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba6a147_13f4_467b_966d_faeb5482199f.slice/crio-de5af8604172eea10072437e566d09721e9268159b7bd7c6e910a6c70ab9a6b5 WatchSource:0}: Error finding container de5af8604172eea10072437e566d09721e9268159b7bd7c6e910a6c70ab9a6b5: Status 404 returned error can't find the container with id de5af8604172eea10072437e566d09721e9268159b7bd7c6e910a6c70ab9a6b5 Jul 17 21:24:37.618171 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:37.618144 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" event={"ID":"0cec39d3-dd08-4376-a02c-a1d2e18c1b5d","Type":"ContainerStarted","Data":"3e80fed384a848c081db8cf8808b00a6ffaaf7eddc7313e5ab1e1ee8bc339668"} Jul 17 21:24:38.625052 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:38.625007 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" event={"ID":"5ba6a147-13f4-467b-966d-faeb5482199f","Type":"ContainerStarted","Data":"de5af8604172eea10072437e566d09721e9268159b7bd7c6e910a6c70ab9a6b5"} Jul 17 21:24:39.631029 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:39.630996 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" event={"ID":"0cec39d3-dd08-4376-a02c-a1d2e18c1b5d","Type":"ContainerStarted","Data":"ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e"} Jul 17 21:24:39.631500 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:39.631102 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" podUID="0cec39d3-dd08-4376-a02c-a1d2e18c1b5d" containerName="registry-server" containerID="cri-o://ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e" gracePeriod=2 Jul 17 21:24:39.632563 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:39.632512 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" event={"ID":"5ba6a147-13f4-467b-966d-faeb5482199f","Type":"ContainerStarted","Data":"184c5d73627c82bea4337c19592f1a30fbe70fe09b1056519646f0b4b1090bfc"} Jul 17 21:24:39.654441 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:39.654398 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" podStartSLOduration=1.540040003 podStartE2EDuration="3.654384053s" totalImagesPullingTime="2.11434405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:24:39.653064192 +0000 UTC m=+460.422412152" watchObservedRunningTime="2026-07-17 21:24:39.654384053 +0000 UTC m=+460.423732043" Jul 17 21:24:39.670235 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:39.670192 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" podStartSLOduration=1.125867436 podStartE2EDuration="2.670178892s" totalImagesPullingTime="1.544311456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:24:39.669467111 +0000 UTC m=+460.438815083" watchObservedRunningTime="2026-07-17 21:24:39.670178892 +0000 UTC m=+460.439526880" Jul 17 21:24:39.869139 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:39.869117 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" Jul 17 21:24:39.926326 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:39.926251 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp82n\" (UniqueName: \"kubernetes.io/projected/0cec39d3-dd08-4376-a02c-a1d2e18c1b5d-kube-api-access-qp82n\") pod \"0cec39d3-dd08-4376-a02c-a1d2e18c1b5d\" (UID: \"0cec39d3-dd08-4376-a02c-a1d2e18c1b5d\") " Jul 17 21:24:39.928463 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:39.928436 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cec39d3-dd08-4376-a02c-a1d2e18c1b5d-kube-api-access-qp82n" (OuterVolumeSpecName: "kube-api-access-qp82n") pod "0cec39d3-dd08-4376-a02c-a1d2e18c1b5d" (UID: "0cec39d3-dd08-4376-a02c-a1d2e18c1b5d"). InnerVolumeSpecName "kube-api-access-qp82n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 21:24:40.027355 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:40.027326 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp82n\" (UniqueName: \"kubernetes.io/projected/0cec39d3-dd08-4376-a02c-a1d2e18c1b5d-kube-api-access-qp82n\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:24:40.637791 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:40.637757 2554 generic.go:356] "Generic (PLEG): container finished" podID="0cec39d3-dd08-4376-a02c-a1d2e18c1b5d" containerID="ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e" exitCode=0 Jul 17 21:24:40.637791 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:40.637796 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" Jul 17 21:24:40.638238 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:40.637845 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" event={"ID":"0cec39d3-dd08-4376-a02c-a1d2e18c1b5d","Type":"ContainerDied","Data":"ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e"} Jul 17 21:24:40.638238 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:40.637888 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d2gd2" event={"ID":"0cec39d3-dd08-4376-a02c-a1d2e18c1b5d","Type":"ContainerDied","Data":"3e80fed384a848c081db8cf8808b00a6ffaaf7eddc7313e5ab1e1ee8bc339668"} Jul 17 21:24:40.638238 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:40.637906 2554 scope.go:117] "RemoveContainer" containerID="ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e" Jul 17 21:24:40.646886 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:40.646868 2554 scope.go:117] "RemoveContainer" containerID="ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e" Jul 17 21:24:40.647147 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:24:40.647129 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e\": container with ID starting with ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e not found: ID does not exist" containerID="ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e" Jul 17 21:24:40.647200 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:40.647156 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e"} err="failed to get container status \"ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e\": rpc error: code = NotFound desc = could not find container \"ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e\": container with ID starting with ac3634e960f3391d628491feb1a7ccf93cc9644758063f7d15bd2a7d6ac2d04e not found: ID does not exist" Jul 17 21:24:40.658350 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:40.658331 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d2gd2"] Jul 17 21:24:40.663250 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:40.663222 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d2gd2"] Jul 17 21:24:41.719091 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:41.719065 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cec39d3-dd08-4376-a02c-a1d2e18c1b5d" path="/var/lib/kubelet/pods/0cec39d3-dd08-4376-a02c-a1d2e18c1b5d/volumes" Jul 17 21:24:47.476124 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:47.476092 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" Jul 17 21:24:47.476507 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:47.476170 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" Jul 17 21:24:47.498982 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:47.498954 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" Jul 17 21:24:47.690339 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:24:47.690314 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-t2z5f" Jul 17 21:25:00.233496 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:00.233459 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:02.850243 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:02.850208 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf"] Jul 17 21:25:02.850634 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:02.850540 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cec39d3-dd08-4376-a02c-a1d2e18c1b5d" containerName="registry-server" Jul 17 21:25:02.850634 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:02.850568 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cec39d3-dd08-4376-a02c-a1d2e18c1b5d" containerName="registry-server" Jul 17 21:25:02.850634 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:02.850628 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cec39d3-dd08-4376-a02c-a1d2e18c1b5d" containerName="registry-server" Jul 17 21:25:02.854787 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:02.854750 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:02.858423 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:02.858393 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-fqtx5\"" Jul 17 21:25:02.871039 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:02.871014 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf"] Jul 17 21:25:03.015988 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:03.015952 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbc6b\" (UniqueName: \"kubernetes.io/projected/e3b681e1-66d8-4c34-a20b-8d56547aa95d-kube-api-access-mbc6b\") pod \"limitador-operator-controller-manager-859d7b78bd-ptcjf\" (UID: \"e3b681e1-66d8-4c34-a20b-8d56547aa95d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:03.116531 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:03.116449 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbc6b\" (UniqueName: \"kubernetes.io/projected/e3b681e1-66d8-4c34-a20b-8d56547aa95d-kube-api-access-mbc6b\") pod \"limitador-operator-controller-manager-859d7b78bd-ptcjf\" (UID: \"e3b681e1-66d8-4c34-a20b-8d56547aa95d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:03.125895 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:03.125872 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbc6b\" (UniqueName: \"kubernetes.io/projected/e3b681e1-66d8-4c34-a20b-8d56547aa95d-kube-api-access-mbc6b\") pod \"limitador-operator-controller-manager-859d7b78bd-ptcjf\" (UID: \"e3b681e1-66d8-4c34-a20b-8d56547aa95d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:03.164410 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:03.164385 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:03.296154 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:03.296092 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf"] Jul 17 21:25:03.298723 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:25:03.298699 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3b681e1_66d8_4c34_a20b_8d56547aa95d.slice/crio-de36746317c8c7b97a1a3bf8106d2377115dcbabf3043e0aa43fd04e4d4aa3a7 WatchSource:0}: Error finding container de36746317c8c7b97a1a3bf8106d2377115dcbabf3043e0aa43fd04e4d4aa3a7: Status 404 returned error can't find the container with id de36746317c8c7b97a1a3bf8106d2377115dcbabf3043e0aa43fd04e4d4aa3a7 Jul 17 21:25:03.736981 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:03.736944 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" event={"ID":"e3b681e1-66d8-4c34-a20b-8d56547aa95d","Type":"ContainerStarted","Data":"de36746317c8c7b97a1a3bf8106d2377115dcbabf3043e0aa43fd04e4d4aa3a7"} Jul 17 21:25:04.432471 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:04.432438 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:04.896644 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:04.896613 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:05.001315 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:05.001285 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:05.747045 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:05.747007 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" event={"ID":"e3b681e1-66d8-4c34-a20b-8d56547aa95d","Type":"ContainerStarted","Data":"b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269"} Jul 17 21:25:05.747419 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:05.747108 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:05.770956 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:05.770906 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" podStartSLOduration=2.01542863 podStartE2EDuration="3.770892506s" totalImagesPullingTime="1.755463876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:25:05.769116563 +0000 UTC m=+486.538464538" watchObservedRunningTime="2026-07-17 21:25:05.770892506 +0000 UTC m=+486.540240484" Jul 17 21:25:06.332271 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:06.332238 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx"] Jul 17 21:25:06.335643 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:06.335627 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx" Jul 17 21:25:06.340533 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:06.340511 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jnh6d\"" Jul 17 21:25:06.341035 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:06.341012 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 17 21:25:06.344620 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:06.344598 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4pbn\" (UniqueName: \"kubernetes.io/projected/db5c16d5-a277-41eb-a3ed-4df6f7bb9330-kube-api-access-s4pbn\") pod \"dns-operator-controller-manager-864dd78c4c-5srfx\" (UID: \"db5c16d5-a277-41eb-a3ed-4df6f7bb9330\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx" Jul 17 21:25:06.356116 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:06.356096 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx"] Jul 17 21:25:06.445270 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:06.445238 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4pbn\" (UniqueName: \"kubernetes.io/projected/db5c16d5-a277-41eb-a3ed-4df6f7bb9330-kube-api-access-s4pbn\") pod \"dns-operator-controller-manager-864dd78c4c-5srfx\" (UID: \"db5c16d5-a277-41eb-a3ed-4df6f7bb9330\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx" Jul 17 21:25:06.457196 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:06.457165 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4pbn\" (UniqueName: \"kubernetes.io/projected/db5c16d5-a277-41eb-a3ed-4df6f7bb9330-kube-api-access-s4pbn\") pod \"dns-operator-controller-manager-864dd78c4c-5srfx\" (UID: \"db5c16d5-a277-41eb-a3ed-4df6f7bb9330\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx" Jul 17 21:25:06.646261 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:06.646169 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx" Jul 17 21:25:06.787494 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:06.787466 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx"] Jul 17 21:25:06.790668 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:25:06.790639 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5c16d5_a277_41eb_a3ed_4df6f7bb9330.slice/crio-be25ed89f9a4898578426789c6459ec810e8313f3c95955a20c3c2c72a1ec215 WatchSource:0}: Error finding container be25ed89f9a4898578426789c6459ec810e8313f3c95955a20c3c2c72a1ec215: Status 404 returned error can't find the container with id be25ed89f9a4898578426789c6459ec810e8313f3c95955a20c3c2c72a1ec215 Jul 17 21:25:07.040106 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:07.040075 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:07.759437 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:07.759400 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx" event={"ID":"db5c16d5-a277-41eb-a3ed-4df6f7bb9330","Type":"ContainerStarted","Data":"be25ed89f9a4898578426789c6459ec810e8313f3c95955a20c3c2c72a1ec215"} Jul 17 21:25:08.734228 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:08.734193 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:08.765120 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:08.765097 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx" event={"ID":"db5c16d5-a277-41eb-a3ed-4df6f7bb9330","Type":"ContainerStarted","Data":"1feda0ff793163f18c4bb8dd0d1e2ed0c7af9b1c4a83d38a2b883ac76dd4282c"} Jul 17 21:25:08.765232 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:08.765180 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx" Jul 17 21:25:08.800731 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:08.800690 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx" podStartSLOduration=0.939621622 podStartE2EDuration="2.800675624s" totalImagesPullingTime="1.861054002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:25:08.800252315 +0000 UTC m=+489.569600294" watchObservedRunningTime="2026-07-17 21:25:08.800675624 +0000 UTC m=+489.570023602" Jul 17 21:25:10.109000 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:10.108959 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:11.384942 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.384909 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd"] Jul 17 21:25:11.388364 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.388346 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:11.391870 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.391842 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-phgnj\"" Jul 17 21:25:11.405730 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.405706 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd"] Jul 17 21:25:11.488968 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.488938 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" (UID: \"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:11.489126 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.488983 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c66h\" (UniqueName: \"kubernetes.io/projected/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-kube-api-access-2c66h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" (UID: \"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:11.589873 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.589829 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" (UID: \"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:11.590040 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.589906 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c66h\" (UniqueName: \"kubernetes.io/projected/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-kube-api-access-2c66h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" (UID: \"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:11.590233 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.590213 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" (UID: \"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:11.635735 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.635671 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c66h\" (UniqueName: \"kubernetes.io/projected/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-kube-api-access-2c66h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" (UID: \"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:11.694088 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.694050 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:11.697946 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.697929 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:11.825907 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.825871 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:11.842257 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:11.842237 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd"] Jul 17 21:25:11.844528 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:25:11.844506 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1ddd0a_bc9c_4682_b2a7_1e7448eeef96.slice/crio-f3b3667debf964b0fe2d2a3269b6c3e96da894b918fce48d9a52d512daec4a0a WatchSource:0}: Error finding container f3b3667debf964b0fe2d2a3269b6c3e96da894b918fce48d9a52d512daec4a0a: Status 404 returned error can't find the container with id f3b3667debf964b0fe2d2a3269b6c3e96da894b918fce48d9a52d512daec4a0a Jul 17 21:25:12.634604 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:12.634567 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:12.784049 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:12.784017 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" event={"ID":"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96","Type":"ContainerStarted","Data":"f3b3667debf964b0fe2d2a3269b6c3e96da894b918fce48d9a52d512daec4a0a"} Jul 17 21:25:14.296905 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:14.296874 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:15.583035 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:15.583001 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm"] Jul 17 21:25:15.592514 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:15.592490 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm" Jul 17 21:25:15.595869 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:15.595845 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-7dvmg\"" Jul 17 21:25:15.596931 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:15.596904 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm"] Jul 17 21:25:15.625138 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:15.625112 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92v4g\" (UniqueName: \"kubernetes.io/projected/91a70b87-46f9-4d22-af69-07bf8efef718-kube-api-access-92v4g\") pod \"authorino-operator-75cc8d5cb7-mzfnm\" (UID: \"91a70b87-46f9-4d22-af69-07bf8efef718\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm" Jul 17 21:25:15.725979 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:15.725942 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92v4g\" (UniqueName: \"kubernetes.io/projected/91a70b87-46f9-4d22-af69-07bf8efef718-kube-api-access-92v4g\") pod \"authorino-operator-75cc8d5cb7-mzfnm\" (UID: \"91a70b87-46f9-4d22-af69-07bf8efef718\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm" Jul 17 21:25:15.737609 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:15.737589 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92v4g\" (UniqueName: \"kubernetes.io/projected/91a70b87-46f9-4d22-af69-07bf8efef718-kube-api-access-92v4g\") pod \"authorino-operator-75cc8d5cb7-mzfnm\" (UID: \"91a70b87-46f9-4d22-af69-07bf8efef718\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm" Jul 17 21:25:15.904517 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:15.904441 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm" Jul 17 21:25:16.294964 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:16.294914 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:16.396609 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:16.396575 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:16.498579 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:16.498522 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:16.755886 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:16.755814 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:16.836954 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:16.836921 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:16.935363 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:16.935333 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:17.201150 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:17.201127 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm"] Jul 17 21:25:17.204265 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:25:17.204242 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a70b87_46f9_4d22_af69_07bf8efef718.slice/crio-cc3d858e3d207f9a0105dd4882d4fc68d38ad71c2a34f39eb0d716f339028633 WatchSource:0}: Error finding container cc3d858e3d207f9a0105dd4882d4fc68d38ad71c2a34f39eb0d716f339028633: Status 404 returned error can't find the container with id cc3d858e3d207f9a0105dd4882d4fc68d38ad71c2a34f39eb0d716f339028633 Jul 17 21:25:17.811337 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:17.811301 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" event={"ID":"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96","Type":"ContainerStarted","Data":"055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d"} Jul 17 21:25:17.811847 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:17.811456 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:17.813375 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:17.813344 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm" event={"ID":"91a70b87-46f9-4d22-af69-07bf8efef718","Type":"ContainerStarted","Data":"cc3d858e3d207f9a0105dd4882d4fc68d38ad71c2a34f39eb0d716f339028633"} Jul 17 21:25:17.833723 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:17.833678 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" podStartSLOduration=1.570227257 podStartE2EDuration="6.833665136s" totalImagesPullingTime="5.263437879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:25:17.831468284 +0000 UTC m=+498.600816262" watchObservedRunningTime="2026-07-17 21:25:17.833665136 +0000 UTC m=+498.603013115" Jul 17 21:25:18.645612 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:18.645577 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:18.743010 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:18.742973 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:19.772164 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:19.772140 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5srfx" Jul 17 21:25:19.832298 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:19.832265 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm" event={"ID":"91a70b87-46f9-4d22-af69-07bf8efef718","Type":"ContainerStarted","Data":"49c1825ac64ac95fcf97f1987f425f00fd6cb3371d29465eb43b97aafda6b13f"} Jul 17 21:25:19.832478 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:19.832411 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm" Jul 17 21:25:19.850190 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:19.850146 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm" podStartSLOduration=2.733618078 podStartE2EDuration="4.85013244s" totalImagesPullingTime="2.116514362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:25:19.849444799 +0000 UTC m=+500.618792778" watchObservedRunningTime="2026-07-17 21:25:19.85013244 +0000 UTC m=+500.619480418" Jul 17 21:25:20.148359 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.148285 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f"] Jul 17 21:25:20.151653 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.151632 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" Jul 17 21:25:20.155685 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.155659 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 17 21:25:20.155685 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.155684 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 17 21:25:20.155839 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.155753 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mn5tn\"" Jul 17 21:25:20.162059 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.162038 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f"] Jul 17 21:25:20.181295 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.181274 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:20.267169 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.267133 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8832db76-f54a-4b86-a1bb-e869385310b2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-n9l8f\" (UID: \"8832db76-f54a-4b86-a1bb-e869385310b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" Jul 17 21:25:20.267340 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.267179 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2hdj\" (UniqueName: \"kubernetes.io/projected/8832db76-f54a-4b86-a1bb-e869385310b2-kube-api-access-p2hdj\") pod \"kuadrant-console-plugin-66b5774985-n9l8f\" (UID: \"8832db76-f54a-4b86-a1bb-e869385310b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" Jul 17 21:25:20.267340 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.267275 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8832db76-f54a-4b86-a1bb-e869385310b2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-n9l8f\" (UID: \"8832db76-f54a-4b86-a1bb-e869385310b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" Jul 17 21:25:20.367692 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.367657 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8832db76-f54a-4b86-a1bb-e869385310b2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-n9l8f\" (UID: \"8832db76-f54a-4b86-a1bb-e869385310b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" Jul 17 21:25:20.367849 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.367746 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8832db76-f54a-4b86-a1bb-e869385310b2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-n9l8f\" (UID: \"8832db76-f54a-4b86-a1bb-e869385310b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" Jul 17 21:25:20.367849 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.367788 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hdj\" (UniqueName: \"kubernetes.io/projected/8832db76-f54a-4b86-a1bb-e869385310b2-kube-api-access-p2hdj\") pod \"kuadrant-console-plugin-66b5774985-n9l8f\" (UID: \"8832db76-f54a-4b86-a1bb-e869385310b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" Jul 17 21:25:20.368446 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.368426 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8832db76-f54a-4b86-a1bb-e869385310b2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-n9l8f\" (UID: \"8832db76-f54a-4b86-a1bb-e869385310b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" Jul 17 21:25:20.370404 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.370368 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8832db76-f54a-4b86-a1bb-e869385310b2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-n9l8f\" (UID: \"8832db76-f54a-4b86-a1bb-e869385310b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" Jul 17 21:25:20.377195 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.377173 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2hdj\" (UniqueName: \"kubernetes.io/projected/8832db76-f54a-4b86-a1bb-e869385310b2-kube-api-access-p2hdj\") pod \"kuadrant-console-plugin-66b5774985-n9l8f\" (UID: \"8832db76-f54a-4b86-a1bb-e869385310b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" Jul 17 21:25:20.401942 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.401872 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:20.461454 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.461427 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" Jul 17 21:25:20.594944 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.594922 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f"] Jul 17 21:25:20.596904 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:25:20.596877 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8832db76_f54a_4b86_a1bb_e869385310b2.slice/crio-7c278054103293f3499a428fd43769bd9b3e7331231a9dd4ad5fd63db8ef7de9 WatchSource:0}: Error finding container 7c278054103293f3499a428fd43769bd9b3e7331231a9dd4ad5fd63db8ef7de9: Status 404 returned error can't find the container with id 7c278054103293f3499a428fd43769bd9b3e7331231a9dd4ad5fd63db8ef7de9 Jul 17 21:25:20.837768 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:20.837739 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" event={"ID":"8832db76-f54a-4b86-a1bb-e869385310b2","Type":"ContainerStarted","Data":"7c278054103293f3499a428fd43769bd9b3e7331231a9dd4ad5fd63db8ef7de9"} Jul 17 21:25:23.821909 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:23.821855 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:23.896260 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:23.896032 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:28.821401 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:28.821367 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:30.573289 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.573255 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd"] Jul 17 21:25:30.573881 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.573443 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" containerName="manager" containerID="cri-o://055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d" gracePeriod=2 Jul 17 21:25:30.585637 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.585575 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd"] Jul 17 21:25:30.588802 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:30.588760 2554 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\": pods \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:30.600331 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.600300 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf"] Jul 17 21:25:30.600761 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.600568 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" containerName="manager" containerID="cri-o://b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269" gracePeriod=2 Jul 17 21:25:30.606077 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.604308 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx"] Jul 17 21:25:30.606077 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.604816 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" containerName="manager" Jul 17 21:25:30.606077 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.604834 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" containerName="manager" Jul 17 21:25:30.606077 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.604954 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" containerName="manager" Jul 17 21:25:30.608269 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:30.608134 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ptcjf\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:30.611154 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:30.611115 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:30.615300 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.615239 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf"] Jul 17 21:25:30.615393 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.615309 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:30.619428 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.619402 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx"] Jul 17 21:25:30.649290 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:30.649239 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:30.651336 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:30.651299 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ptcjf\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:30.659738 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.659713 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lbbl\" (UniqueName: \"kubernetes.io/projected/671a3300-1a30-4fcc-a214-05e43987e153-kube-api-access-2lbbl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-26vlx\" (UID: \"671a3300-1a30-4fcc-a214-05e43987e153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:30.659870 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.659766 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/671a3300-1a30-4fcc-a214-05e43987e153-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-26vlx\" (UID: \"671a3300-1a30-4fcc-a214-05e43987e153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:30.760478 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.760440 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lbbl\" (UniqueName: \"kubernetes.io/projected/671a3300-1a30-4fcc-a214-05e43987e153-kube-api-access-2lbbl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-26vlx\" (UID: \"671a3300-1a30-4fcc-a214-05e43987e153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:30.760675 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.760502 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/671a3300-1a30-4fcc-a214-05e43987e153-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-26vlx\" (UID: \"671a3300-1a30-4fcc-a214-05e43987e153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:30.760924 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.760896 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/671a3300-1a30-4fcc-a214-05e43987e153-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-26vlx\" (UID: \"671a3300-1a30-4fcc-a214-05e43987e153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:30.767524 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.767495 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n"] Jul 17 21:25:30.767985 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.767949 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" containerName="manager" Jul 17 21:25:30.767985 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.767973 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" containerName="manager" Jul 17 21:25:30.768138 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.768083 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" containerName="manager" Jul 17 21:25:30.775115 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.775093 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:25:30.775910 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.775890 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lbbl\" (UniqueName: \"kubernetes.io/projected/671a3300-1a30-4fcc-a214-05e43987e153-kube-api-access-2lbbl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-26vlx\" (UID: \"671a3300-1a30-4fcc-a214-05e43987e153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:30.777757 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:30.777719 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:30.780181 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:30.780146 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ptcjf\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:30.785577 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.785558 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n"] Jul 17 21:25:30.840536 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.840460 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mzfnm" Jul 17 21:25:30.843259 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:30.843224 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:30.845740 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:30.845705 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ptcjf\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:30.861070 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.861040 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31318d4d-e596-4614-9c2f-331bcc8e00f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9fw5n\" (UID: \"31318d4d-e596-4614-9c2f-331bcc8e00f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:25:30.861205 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.861097 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thctk\" (UniqueName: \"kubernetes.io/projected/31318d4d-e596-4614-9c2f-331bcc8e00f4-kube-api-access-thctk\") pod \"kuadrant-operator-controller-manager-86878c769f-9fw5n\" (UID: \"31318d4d-e596-4614-9c2f-331bcc8e00f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:25:30.961976 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.961938 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31318d4d-e596-4614-9c2f-331bcc8e00f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9fw5n\" (UID: \"31318d4d-e596-4614-9c2f-331bcc8e00f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:25:30.962147 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.961994 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thctk\" (UniqueName: \"kubernetes.io/projected/31318d4d-e596-4614-9c2f-331bcc8e00f4-kube-api-access-thctk\") pod \"kuadrant-operator-controller-manager-86878c769f-9fw5n\" (UID: \"31318d4d-e596-4614-9c2f-331bcc8e00f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:25:30.962323 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.962296 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31318d4d-e596-4614-9c2f-331bcc8e00f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9fw5n\" (UID: \"31318d4d-e596-4614-9c2f-331bcc8e00f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:25:30.976133 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:30.976100 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thctk\" (UniqueName: \"kubernetes.io/projected/31318d4d-e596-4614-9c2f-331bcc8e00f4-kube-api-access-thctk\") pod \"kuadrant-operator-controller-manager-86878c769f-9fw5n\" (UID: \"31318d4d-e596-4614-9c2f-331bcc8e00f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:25:31.034327 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:31.034291 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:31.099348 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:31.099263 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:25:32.444707 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:32.444673 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:32.548385 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:32.548348 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:35.123122 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:35.123031 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:35.230025 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:35.229991 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:38.044501 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:38.044465 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:38.145088 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:38.144940 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:41.841101 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:41.841058 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:41.941120 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:41.941082 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:42.138182 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.138149 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx"] Jul 17 21:25:42.151237 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.151208 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n"] Jul 17 21:25:42.228953 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:25:42.228918 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod671a3300_1a30_4fcc_a214_05e43987e153.slice/crio-f088aa8eede7e07010f039a0cffb70d87c7b52aba73fba7244b13bf20fe5161f WatchSource:0}: Error finding container f088aa8eede7e07010f039a0cffb70d87c7b52aba73fba7244b13bf20fe5161f: Status 404 returned error can't find the container with id f088aa8eede7e07010f039a0cffb70d87c7b52aba73fba7244b13bf20fe5161f Jul 17 21:25:42.229647 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:25:42.229613 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31318d4d_e596_4614_9c2f_331bcc8e00f4.slice/crio-7f37acb6a33660d90db9e2d752acf768f919d2a836825e60829c270e25e53a32 WatchSource:0}: Error finding container 7f37acb6a33660d90db9e2d752acf768f919d2a836825e60829c270e25e53a32: Status 404 returned error can't find the container with id 7f37acb6a33660d90db9e2d752acf768f919d2a836825e60829c270e25e53a32 Jul 17 21:25:42.284179 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.284158 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:42.286619 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:42.286591 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:42.311814 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.311787 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:42.320708 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:42.320677 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:42.326975 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:42.326947 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ptcjf\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:42.365998 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.365944 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c66h\" (UniqueName: \"kubernetes.io/projected/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-kube-api-access-2c66h\") pod \"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96\" (UID: \"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96\") " Jul 17 21:25:42.366094 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.366052 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-extensions-socket-volume\") pod \"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96\" (UID: \"fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96\") " Jul 17 21:25:42.366677 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.366651 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" (UID: "fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 21:25:42.367925 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.367904 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-kube-api-access-2c66h" (OuterVolumeSpecName: "kube-api-access-2c66h") pod "fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" (UID: "fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96"). InnerVolumeSpecName "kube-api-access-2c66h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 21:25:42.467284 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.467255 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbc6b\" (UniqueName: \"kubernetes.io/projected/e3b681e1-66d8-4c34-a20b-8d56547aa95d-kube-api-access-mbc6b\") pod \"e3b681e1-66d8-4c34-a20b-8d56547aa95d\" (UID: \"e3b681e1-66d8-4c34-a20b-8d56547aa95d\") " Jul 17 21:25:42.467439 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.467415 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c66h\" (UniqueName: \"kubernetes.io/projected/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-kube-api-access-2c66h\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:25:42.467439 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.467427 2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96-extensions-socket-volume\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:25:42.469422 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.469389 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3b681e1-66d8-4c34-a20b-8d56547aa95d-kube-api-access-mbc6b" (OuterVolumeSpecName: "kube-api-access-mbc6b") pod "e3b681e1-66d8-4c34-a20b-8d56547aa95d" (UID: "e3b681e1-66d8-4c34-a20b-8d56547aa95d"). InnerVolumeSpecName "kube-api-access-mbc6b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 21:25:42.568218 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.568175 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbc6b\" (UniqueName: \"kubernetes.io/projected/e3b681e1-66d8-4c34-a20b-8d56547aa95d-kube-api-access-mbc6b\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:25:42.969042 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.969011 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" event={"ID":"31318d4d-e596-4614-9c2f-331bcc8e00f4","Type":"ContainerStarted","Data":"9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526"} Jul 17 21:25:42.969470 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.969060 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" event={"ID":"31318d4d-e596-4614-9c2f-331bcc8e00f4","Type":"ContainerStarted","Data":"7f37acb6a33660d90db9e2d752acf768f919d2a836825e60829c270e25e53a32"} Jul 17 21:25:42.969470 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.969122 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:25:42.970349 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.970327 2554 generic.go:356] "Generic (PLEG): container finished" podID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" containerID="055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d" exitCode=0 Jul 17 21:25:42.970512 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.970369 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:42.970512 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.970374 2554 scope.go:117] "RemoveContainer" containerID="055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d" Jul 17 21:25:42.971832 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:42.971747 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ptcjf\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:42.972483 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.972459 2554 generic.go:356] "Generic (PLEG): container finished" podID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" containerID="b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269" exitCode=0 Jul 17 21:25:42.972594 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.972519 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:42.974755 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.974731 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" event={"ID":"8832db76-f54a-4b86-a1bb-e869385310b2","Type":"ContainerStarted","Data":"117871ece2ba1e393c8ef387f9cf6beb03f24d76b7089abd759187db8cf980a0"} Jul 17 21:25:42.976700 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.976678 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" event={"ID":"671a3300-1a30-4fcc-a214-05e43987e153","Type":"ContainerStarted","Data":"24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69"} Jul 17 21:25:42.976808 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.976721 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" event={"ID":"671a3300-1a30-4fcc-a214-05e43987e153","Type":"ContainerStarted","Data":"f088aa8eede7e07010f039a0cffb70d87c7b52aba73fba7244b13bf20fe5161f"} Jul 17 21:25:42.976911 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.976883 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:42.983164 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.982953 2554 scope.go:117] "RemoveContainer" containerID="055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d" Jul 17 21:25:42.983290 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:42.983258 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d\": container with ID starting with 055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d not found: ID does not exist" containerID="055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d" Jul 17 21:25:42.983386 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.983300 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d"} err="failed to get container status \"055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d\": rpc error: code = NotFound desc = could not find container \"055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d\": container with ID starting with 055f342cde489e0094cebd176f28a0d324d2c4b15b85a820ae34c50ef9dfd92d not found: ID does not exist" Jul 17 21:25:42.983386 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.983323 2554 scope.go:117] "RemoveContainer" containerID="b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269" Jul 17 21:25:42.991702 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.991686 2554 scope.go:117] "RemoveContainer" containerID="b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269" Jul 17 21:25:42.991927 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:42.991912 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269\": container with ID starting with b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269 not found: ID does not exist" containerID="b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269" Jul 17 21:25:42.991976 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.991932 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269"} err="failed to get container status \"b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269\": rpc error: code = NotFound desc = could not find container \"b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269\": container with ID starting with b138f14d93f2302ca6b8a34f7ad3413b3810cf840b225dc56e6d558452f6d269 not found: ID does not exist" Jul 17 21:25:42.999844 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:42.996234 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" podStartSLOduration=12.996218214 podStartE2EDuration="12.996218214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:25:42.993797667 +0000 UTC m=+523.763145644" watchObservedRunningTime="2026-07-17 21:25:42.996218214 +0000 UTC m=+523.765566192" Jul 17 21:25:42.999844 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:42.998405 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:43.028813 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:43.028769 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" podStartSLOduration=13.028754903 podStartE2EDuration="13.028754903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:25:43.027705399 +0000 UTC m=+523.797053377" watchObservedRunningTime="2026-07-17 21:25:43.028754903 +0000 UTC m=+523.798102880" Jul 17 21:25:43.045137 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:43.045096 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n9l8f" podStartSLOduration=1.362717767 podStartE2EDuration="23.045073373s" totalImagesPullingTime="21.682355606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:25:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:25:43.043784065 +0000 UTC m=+523.813132046" watchObservedRunningTime="2026-07-17 21:25:43.045073373 +0000 UTC m=+523.814421350" Jul 17 21:25:43.045951 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:43.045929 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xsxbd\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xsxbd" Jul 17 21:25:43.048146 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:43.048125 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ptcjf\" is forbidden: User \"system:node:ip-10-0-131-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-188.ec2.internal' and this object" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ptcjf" Jul 17 21:25:43.719768 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:43.719731 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3b681e1-66d8-4c34-a20b-8d56547aa95d" path="/var/lib/kubelet/pods/e3b681e1-66d8-4c34-a20b-8d56547aa95d/volumes" Jul 17 21:25:43.720168 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:43.720149 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96" path="/var/lib/kubelet/pods/fb1ddd0a-bc9c-4682-b2a7-1e7448eeef96/volumes" Jul 17 21:25:50.476617 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:50.476514 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:50.591957 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:50.591927 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:25:53.986189 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:53.986161 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:53.986596 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:53.986318 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:25:54.064251 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:54.064211 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx"] Jul 17 21:25:54.064411 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:54.064375 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" podUID="671a3300-1a30-4fcc-a214-05e43987e153" containerName="manager" containerID="cri-o://24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69" gracePeriod=10 Jul 17 21:25:54.320409 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:54.320384 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:54.466336 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:54.466302 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/671a3300-1a30-4fcc-a214-05e43987e153-extensions-socket-volume\") pod \"671a3300-1a30-4fcc-a214-05e43987e153\" (UID: \"671a3300-1a30-4fcc-a214-05e43987e153\") " Jul 17 21:25:54.466508 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:54.466424 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lbbl\" (UniqueName: \"kubernetes.io/projected/671a3300-1a30-4fcc-a214-05e43987e153-kube-api-access-2lbbl\") pod \"671a3300-1a30-4fcc-a214-05e43987e153\" (UID: \"671a3300-1a30-4fcc-a214-05e43987e153\") " Jul 17 21:25:54.466783 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:54.466760 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/671a3300-1a30-4fcc-a214-05e43987e153-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "671a3300-1a30-4fcc-a214-05e43987e153" (UID: "671a3300-1a30-4fcc-a214-05e43987e153"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 21:25:54.468587 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:54.468555 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/671a3300-1a30-4fcc-a214-05e43987e153-kube-api-access-2lbbl" (OuterVolumeSpecName: "kube-api-access-2lbbl") pod "671a3300-1a30-4fcc-a214-05e43987e153" (UID: "671a3300-1a30-4fcc-a214-05e43987e153"). InnerVolumeSpecName "kube-api-access-2lbbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 21:25:54.567397 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:54.567317 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lbbl\" (UniqueName: \"kubernetes.io/projected/671a3300-1a30-4fcc-a214-05e43987e153-kube-api-access-2lbbl\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:25:54.567397 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:54.567344 2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/671a3300-1a30-4fcc-a214-05e43987e153-extensions-socket-volume\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:25:55.036341 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:55.036304 2554 generic.go:356] "Generic (PLEG): container finished" podID="671a3300-1a30-4fcc-a214-05e43987e153" containerID="24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69" exitCode=0 Jul 17 21:25:55.036747 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:55.036351 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" Jul 17 21:25:55.036747 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:55.036390 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" event={"ID":"671a3300-1a30-4fcc-a214-05e43987e153","Type":"ContainerDied","Data":"24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69"} Jul 17 21:25:55.036747 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:55.036446 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx" event={"ID":"671a3300-1a30-4fcc-a214-05e43987e153","Type":"ContainerDied","Data":"f088aa8eede7e07010f039a0cffb70d87c7b52aba73fba7244b13bf20fe5161f"} Jul 17 21:25:55.036747 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:55.036468 2554 scope.go:117] "RemoveContainer" containerID="24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69" Jul 17 21:25:55.045761 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:55.045747 2554 scope.go:117] "RemoveContainer" containerID="24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69" Jul 17 21:25:55.045967 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:25:55.045947 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69\": container with ID starting with 24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69 not found: ID does not exist" containerID="24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69" Jul 17 21:25:55.046016 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:55.045982 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69"} err="failed to get container status \"24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69\": rpc error: code = NotFound desc = could not find container \"24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69\": container with ID starting with 24970b0e4b069d17ae4003a5293a4f7d54c14a482a82d1bf5b0fcf6cc0853a69 not found: ID does not exist" Jul 17 21:25:55.059570 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:55.059535 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx"] Jul 17 21:25:55.067983 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:55.065330 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-26vlx"] Jul 17 21:25:55.719516 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:25:55.719482 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="671a3300-1a30-4fcc-a214-05e43987e153" path="/var/lib/kubelet/pods/671a3300-1a30-4fcc-a214-05e43987e153/volumes" Jul 17 21:26:10.343861 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.343830 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5"] Jul 17 21:26:10.344338 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.344173 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="671a3300-1a30-4fcc-a214-05e43987e153" containerName="manager" Jul 17 21:26:10.344338 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.344184 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="671a3300-1a30-4fcc-a214-05e43987e153" containerName="manager" Jul 17 21:26:10.344338 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.344249 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="671a3300-1a30-4fcc-a214-05e43987e153" containerName="manager" Jul 17 21:26:10.370354 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.370325 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:10.516476 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.516433 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:10.532512 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.532485 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5"] Jul 17 21:26:10.532658 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.532620 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.535498 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.535480 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-q8sj6\"" Jul 17 21:26:10.587734 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.587679 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.587734 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.587727 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.587929 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.587780 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnx9q\" (UniqueName: \"kubernetes.io/projected/2c68926a-c530-4f68-ab8c-4e34730f5664-kube-api-access-jnx9q\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.587929 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.587819 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.587929 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.587845 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c68926a-c530-4f68-ab8c-4e34730f5664-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.587929 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.587919 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.588133 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.587966 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.588133 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.588026 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.588133 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.588052 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.688806 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.688724 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.688806 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.688758 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689020 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.688805 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689020 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.688838 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689020 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.688869 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnx9q\" (UniqueName: \"kubernetes.io/projected/2c68926a-c530-4f68-ab8c-4e34730f5664-kube-api-access-jnx9q\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689020 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.688900 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689020 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.688936 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c68926a-c530-4f68-ab8c-4e34730f5664-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689020 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.688980 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689020 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.689020 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689370 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.689259 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689370 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.689280 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689455 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.689429 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689497 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.689458 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.689818 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.689781 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c68926a-c530-4f68-ab8c-4e34730f5664-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.691435 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.691416 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.691645 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.691625 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.697625 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.697604 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c68926a-c530-4f68-ab8c-4e34730f5664-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.697993 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.697970 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnx9q\" (UniqueName: \"kubernetes.io/projected/2c68926a-c530-4f68-ab8c-4e34730f5664-kube-api-access-jnx9q\") pod \"maas-default-gateway-openshift-default-56654dcf7-gkjn5\" (UID: \"2c68926a-c530-4f68-ab8c-4e34730f5664\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:10.843355 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:10.843325 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:11.076591 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:11.076561 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5"] Jul 17 21:26:11.078052 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:26:11.078024 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c68926a_c530_4f68_ab8c_4e34730f5664.slice/crio-ad8b97d09464ca4926945b002834ea1c850cb0949fe5e6ee6261ca11d6c153ea WatchSource:0}: Error finding container ad8b97d09464ca4926945b002834ea1c850cb0949fe5e6ee6261ca11d6c153ea: Status 404 returned error can't find the container with id ad8b97d09464ca4926945b002834ea1c850cb0949fe5e6ee6261ca11d6c153ea Jul 17 21:26:11.080442 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:11.080401 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 17 21:26:11.080538 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:11.080488 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 17 21:26:11.080611 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:11.080532 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 17 21:26:11.111594 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:11.111570 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" event={"ID":"2c68926a-c530-4f68-ab8c-4e34730f5664","Type":"ContainerStarted","Data":"ad8b97d09464ca4926945b002834ea1c850cb0949fe5e6ee6261ca11d6c153ea"} Jul 17 21:26:12.118133 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:12.118093 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" event={"ID":"2c68926a-c530-4f68-ab8c-4e34730f5664","Type":"ContainerStarted","Data":"c5fb5c43ecec4855204141cc7e9e9639ee7707ae8c4da25a516dc8b5a93d54d5"} Jul 17 21:26:12.140362 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:12.140308 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" podStartSLOduration=2.140293956 podStartE2EDuration="2.140293956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:26:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:26:12.139286167 +0000 UTC m=+552.908634145" watchObservedRunningTime="2026-07-17 21:26:12.140293956 +0000 UTC m=+552.909641979" Jul 17 21:26:12.844189 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:12.844158 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:12.849004 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:12.848981 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:13.091262 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.091222 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:13.123228 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.123150 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:13.124032 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.124017 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:26:13.190596 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.190563 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:13.227381 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.227351 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:26:13.231707 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.231689 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" Jul 17 21:26:13.234464 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.234448 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 17 21:26:13.241038 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.241014 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:26:13.269236 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.269207 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:26:13.271258 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.271230 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:13.306236 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.306206 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1d99b42c-a11c-4469-8ce9-db57c07cf719-config-file\") pod \"limitador-limitador-54d6d988d5-vksmg\" (UID: \"1d99b42c-a11c-4469-8ce9-db57c07cf719\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" Jul 17 21:26:13.306378 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.306264 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqsm4\" (UniqueName: \"kubernetes.io/projected/1d99b42c-a11c-4469-8ce9-db57c07cf719-kube-api-access-dqsm4\") pod \"limitador-limitador-54d6d988d5-vksmg\" (UID: \"1d99b42c-a11c-4469-8ce9-db57c07cf719\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" Jul 17 21:26:13.407057 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.406976 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1d99b42c-a11c-4469-8ce9-db57c07cf719-config-file\") pod \"limitador-limitador-54d6d988d5-vksmg\" (UID: \"1d99b42c-a11c-4469-8ce9-db57c07cf719\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" Jul 17 21:26:13.407196 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.407062 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqsm4\" (UniqueName: \"kubernetes.io/projected/1d99b42c-a11c-4469-8ce9-db57c07cf719-kube-api-access-dqsm4\") pod \"limitador-limitador-54d6d988d5-vksmg\" (UID: \"1d99b42c-a11c-4469-8ce9-db57c07cf719\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" Jul 17 21:26:13.407701 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.407682 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1d99b42c-a11c-4469-8ce9-db57c07cf719-config-file\") pod \"limitador-limitador-54d6d988d5-vksmg\" (UID: \"1d99b42c-a11c-4469-8ce9-db57c07cf719\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" Jul 17 21:26:13.415936 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.415913 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqsm4\" (UniqueName: \"kubernetes.io/projected/1d99b42c-a11c-4469-8ce9-db57c07cf719-kube-api-access-dqsm4\") pod \"limitador-limitador-54d6d988d5-vksmg\" (UID: \"1d99b42c-a11c-4469-8ce9-db57c07cf719\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" Jul 17 21:26:13.542628 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.542594 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" Jul 17 21:26:13.677494 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.677469 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:26:13.678866 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:26:13.678834 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d99b42c_a11c_4469_8ce9_db57c07cf719.slice/crio-eb1a2b2ca93b31703b6b9e6cf96a748d07f6d30c1f811637918b6f2d433c4a55 WatchSource:0}: Error finding container eb1a2b2ca93b31703b6b9e6cf96a748d07f6d30c1f811637918b6f2d433c4a55: Status 404 returned error can't find the container with id eb1a2b2ca93b31703b6b9e6cf96a748d07f6d30c1f811637918b6f2d433c4a55 Jul 17 21:26:13.917621 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.917567 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:13.988683 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:13.988577 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:14.067221 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:14.067190 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-m6mfd"] Jul 17 21:26:14.069839 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:14.069822 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m6mfd" Jul 17 21:26:14.082279 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:14.082258 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rlptf\"" Jul 17 21:26:14.088765 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:14.088743 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m6mfd"] Jul 17 21:26:14.111563 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:14.111523 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxpx6\" (UniqueName: \"kubernetes.io/projected/2d3a45bb-7f18-432c-b933-115d6d27024c-kube-api-access-vxpx6\") pod \"authorino-75d556bd77-m6mfd\" (UID: \"2d3a45bb-7f18-432c-b933-115d6d27024c\") " pod="kuadrant-system/authorino-75d556bd77-m6mfd" Jul 17 21:26:14.128683 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:14.128658 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" event={"ID":"1d99b42c-a11c-4469-8ce9-db57c07cf719","Type":"ContainerStarted","Data":"eb1a2b2ca93b31703b6b9e6cf96a748d07f6d30c1f811637918b6f2d433c4a55"} Jul 17 21:26:14.215111 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:14.215077 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxpx6\" (UniqueName: \"kubernetes.io/projected/2d3a45bb-7f18-432c-b933-115d6d27024c-kube-api-access-vxpx6\") pod \"authorino-75d556bd77-m6mfd\" (UID: \"2d3a45bb-7f18-432c-b933-115d6d27024c\") " pod="kuadrant-system/authorino-75d556bd77-m6mfd" Jul 17 21:26:14.228536 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:14.228486 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxpx6\" (UniqueName: \"kubernetes.io/projected/2d3a45bb-7f18-432c-b933-115d6d27024c-kube-api-access-vxpx6\") pod \"authorino-75d556bd77-m6mfd\" (UID: \"2d3a45bb-7f18-432c-b933-115d6d27024c\") " pod="kuadrant-system/authorino-75d556bd77-m6mfd" Jul 17 21:26:14.379479 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:14.379444 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m6mfd" Jul 17 21:26:14.559349 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:14.559319 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m6mfd"] Jul 17 21:26:14.562878 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:26:14.562832 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d3a45bb_7f18_432c_b933_115d6d27024c.slice/crio-bebf91365738a215886139407c38303c5726c36d17cca1ea80ac7fca3b8b9c9e WatchSource:0}: Error finding container bebf91365738a215886139407c38303c5726c36d17cca1ea80ac7fca3b8b9c9e: Status 404 returned error can't find the container with id bebf91365738a215886139407c38303c5726c36d17cca1ea80ac7fca3b8b9c9e Jul 17 21:26:15.135341 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:15.135301 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m6mfd" event={"ID":"2d3a45bb-7f18-432c-b933-115d6d27024c","Type":"ContainerStarted","Data":"bebf91365738a215886139407c38303c5726c36d17cca1ea80ac7fca3b8b9c9e"} Jul 17 21:26:15.752741 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:15.752714 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:15.828896 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:15.828863 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:17.149764 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:17.149731 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" event={"ID":"1d99b42c-a11c-4469-8ce9-db57c07cf719","Type":"ContainerStarted","Data":"24a86e3e47381a4bd48001957eb3ef26c6c25886cde5c641fc3b263211482871"} Jul 17 21:26:17.150263 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:17.149881 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" Jul 17 21:26:17.177335 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:17.177277 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" podStartSLOduration=1.56221296 podStartE2EDuration="4.177255731s" totalImagesPullingTime="2.615042771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:26:17.175368342 +0000 UTC m=+557.944716321" watchObservedRunningTime="2026-07-17 21:26:17.177255731 +0000 UTC m=+557.946603781" Jul 17 21:26:19.161780 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:19.161747 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m6mfd" event={"ID":"2d3a45bb-7f18-432c-b933-115d6d27024c","Type":"ContainerStarted","Data":"e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487"} Jul 17 21:26:19.177753 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:19.177704 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-m6mfd" podStartSLOduration=1.086530922 podStartE2EDuration="5.177688081s" totalImagesPullingTime="4.091157159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:26:19.1771476 +0000 UTC m=+559.946495590" watchObservedRunningTime="2026-07-17 21:26:19.177688081 +0000 UTC m=+559.947036086" Jul 17 21:26:28.155699 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:28.155667 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-vksmg" Jul 17 21:26:29.191921 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:29.191881 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:29.286754 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:29.286697 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:29.614050 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:29.614018 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:41.252719 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:41.252685 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:41.761189 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:41.761158 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:47.017283 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.017243 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m6mfd"] Jul 17 21:26:47.017877 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.017437 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-m6mfd" podUID="2d3a45bb-7f18-432c-b933-115d6d27024c" containerName="authorino" containerID="cri-o://e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487" gracePeriod=30 Jul 17 21:26:47.262868 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.262842 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m6mfd" Jul 17 21:26:47.292263 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.292234 2554 generic.go:356] "Generic (PLEG): container finished" podID="2d3a45bb-7f18-432c-b933-115d6d27024c" containerID="e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487" exitCode=0 Jul 17 21:26:47.292386 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.292269 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m6mfd" Jul 17 21:26:47.292386 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.292316 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m6mfd" event={"ID":"2d3a45bb-7f18-432c-b933-115d6d27024c","Type":"ContainerDied","Data":"e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487"} Jul 17 21:26:47.292386 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.292358 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m6mfd" event={"ID":"2d3a45bb-7f18-432c-b933-115d6d27024c","Type":"ContainerDied","Data":"bebf91365738a215886139407c38303c5726c36d17cca1ea80ac7fca3b8b9c9e"} Jul 17 21:26:47.292386 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.292375 2554 scope.go:117] "RemoveContainer" containerID="e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487" Jul 17 21:26:47.301673 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.301657 2554 scope.go:117] "RemoveContainer" containerID="e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487" Jul 17 21:26:47.301934 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:26:47.301917 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487\": container with ID starting with e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487 not found: ID does not exist" containerID="e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487" Jul 17 21:26:47.301976 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.301945 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487"} err="failed to get container status \"e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487\": rpc error: code = NotFound desc = could not find container \"e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487\": container with ID starting with e284854243c66f6fce08210290599703a1fcda3fbf659778c53292f94f347487 not found: ID does not exist" Jul 17 21:26:47.409345 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.409315 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxpx6\" (UniqueName: \"kubernetes.io/projected/2d3a45bb-7f18-432c-b933-115d6d27024c-kube-api-access-vxpx6\") pod \"2d3a45bb-7f18-432c-b933-115d6d27024c\" (UID: \"2d3a45bb-7f18-432c-b933-115d6d27024c\") " Jul 17 21:26:47.411507 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.411477 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d3a45bb-7f18-432c-b933-115d6d27024c-kube-api-access-vxpx6" (OuterVolumeSpecName: "kube-api-access-vxpx6") pod "2d3a45bb-7f18-432c-b933-115d6d27024c" (UID: "2d3a45bb-7f18-432c-b933-115d6d27024c"). InnerVolumeSpecName "kube-api-access-vxpx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 21:26:47.509956 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.509930 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxpx6\" (UniqueName: \"kubernetes.io/projected/2d3a45bb-7f18-432c-b933-115d6d27024c-kube-api-access-vxpx6\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:26:47.613757 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.613731 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m6mfd"] Jul 17 21:26:47.619290 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.619267 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m6mfd"] Jul 17 21:26:47.719836 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:47.719809 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d3a45bb-7f18-432c-b933-115d6d27024c" path="/var/lib/kubelet/pods/2d3a45bb-7f18-432c-b933-115d6d27024c/volumes" Jul 17 21:26:51.593409 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:51.593386 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:51.693941 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:51.693910 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:51.765836 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:51.765802 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:55.235740 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:55.235705 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:55.333411 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:55.333373 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:58.262636 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:58.262601 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:58.364173 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:58.364133 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:26:59.598405 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:59.598380 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:26:59.605472 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:26:59.605450 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:27:27.893562 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:27.893462 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:27:29.539576 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:29.539509 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:27:30.471539 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:30.471499 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:27:36.423350 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:36.423315 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:27:37.535581 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:37.535537 2554 generic.go:356] "Generic (PLEG): container finished" podID="2c68926a-c530-4f68-ab8c-4e34730f5664" containerID="c5fb5c43ecec4855204141cc7e9e9639ee7707ae8c4da25a516dc8b5a93d54d5" exitCode=0 Jul 17 21:27:37.535980 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:37.535609 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" event={"ID":"2c68926a-c530-4f68-ab8c-4e34730f5664","Type":"ContainerDied","Data":"c5fb5c43ecec4855204141cc7e9e9639ee7707ae8c4da25a516dc8b5a93d54d5"} Jul 17 21:27:37.536022 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:37.535983 2554 scope.go:117] "RemoveContainer" containerID="c5fb5c43ecec4855204141cc7e9e9639ee7707ae8c4da25a516dc8b5a93d54d5" Jul 17 21:27:37.536931 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:37.536902 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 17 21:27:37.537021 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:37.536987 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 17 21:27:37.537083 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:37.537044 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 17 21:27:38.541872 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:38.541835 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" event={"ID":"2c68926a-c530-4f68-ab8c-4e34730f5664","Type":"ContainerStarted","Data":"9ccec0aa9fa7e38b5e53ce46768dce602fefc49eb9004857591645e3942445ed"} Jul 17 21:27:38.802344 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:38.802251 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:27:38.843726 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:38.843692 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:27:38.844887 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:38.844860 2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gkjn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 17 21:27:38.845004 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:38.844921 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" podUID="2c68926a-c530-4f68-ab8c-4e34730f5664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 17 21:27:38.901289 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:38.901249 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:27:39.843879 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:39.843848 2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gkjn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 17 21:27:39.844251 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:39.843912 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" podUID="2c68926a-c530-4f68-ab8c-4e34730f5664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 17 21:27:40.843881 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:40.843849 2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gkjn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 17 21:27:40.844261 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:40.843901 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" podUID="2c68926a-c530-4f68-ab8c-4e34730f5664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 17 21:27:40.974144 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:40.974109 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:27:41.844823 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:41.844779 2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gkjn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 17 21:27:41.845299 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:41.844841 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" podUID="2c68926a-c530-4f68-ab8c-4e34730f5664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 17 21:27:42.869729 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:42.869698 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:27:42.870184 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:42.870076 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:27:42.870842 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:42.870821 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gkjn5" Jul 17 21:27:44.548586 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:44.548533 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:27:44.977824 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:27:44.977736 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:28:01.382394 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:01.382355 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:28:02.240465 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:02.240433 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:28:06.938839 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.938808 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn"] Jul 17 21:28:06.939274 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.939217 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d3a45bb-7f18-432c-b933-115d6d27024c" containerName="authorino" Jul 17 21:28:06.939274 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.939229 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d3a45bb-7f18-432c-b933-115d6d27024c" containerName="authorino" Jul 17 21:28:06.939378 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.939329 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d3a45bb-7f18-432c-b933-115d6d27024c" containerName="authorino" Jul 17 21:28:06.942406 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.942388 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:06.945291 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.945269 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 17 21:28:06.951094 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.946911 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 17 21:28:06.951094 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.946979 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z79wp\"" Jul 17 21:28:06.951094 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.947534 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 17 21:28:06.957384 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.957356 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn"] Jul 17 21:28:06.979183 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.979159 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:06.979332 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.979208 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:06.979332 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.979292 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:06.979332 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.979324 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:06.979495 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.979374 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/745ea714-346c-467b-88ca-d1305f634685-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:06.979495 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:06.979421 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmst4\" (UniqueName: \"kubernetes.io/projected/745ea714-346c-467b-88ca-d1305f634685-kube-api-access-bmst4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.045270 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.045227 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:28:07.080415 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.080381 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.080415 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.080415 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.080671 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.080445 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/745ea714-346c-467b-88ca-d1305f634685-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.080671 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.080470 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmst4\" (UniqueName: \"kubernetes.io/projected/745ea714-346c-467b-88ca-d1305f634685-kube-api-access-bmst4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.080671 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.080511 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.080671 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.080567 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.080876 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.080844 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.081005 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.080988 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.081203 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.081156 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.082868 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.082843 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/745ea714-346c-467b-88ca-d1305f634685-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.083158 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.083142 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/745ea714-346c-467b-88ca-d1305f634685-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.088211 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.088186 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmst4\" (UniqueName: \"kubernetes.io/projected/745ea714-346c-467b-88ca-d1305f634685-kube-api-access-bmst4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlkpn\" (UID: \"745ea714-346c-467b-88ca-d1305f634685\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.259375 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.259282 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:07.401184 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.401131 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn"] Jul 17 21:28:07.403633 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:28:07.403602 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod745ea714_346c_467b_88ca_d1305f634685.slice/crio-5331a4b0cdc5cca82a785d5cc2f47a89354c0a6a990c21c3bca498bec845e7c2 WatchSource:0}: Error finding container 5331a4b0cdc5cca82a785d5cc2f47a89354c0a6a990c21c3bca498bec845e7c2: Status 404 returned error can't find the container with id 5331a4b0cdc5cca82a785d5cc2f47a89354c0a6a990c21c3bca498bec845e7c2 Jul 17 21:28:07.677573 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:07.677517 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" event={"ID":"745ea714-346c-467b-88ca-d1305f634685","Type":"ContainerStarted","Data":"5331a4b0cdc5cca82a785d5cc2f47a89354c0a6a990c21c3bca498bec845e7c2"} Jul 17 21:28:08.337110 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:08.337071 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:28:08.684343 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:08.684244 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" event={"ID":"745ea714-346c-467b-88ca-d1305f634685","Type":"ContainerStarted","Data":"8638378094efcd18d72caf938ba625f1f10e160f00008641621ca35c79737ec7"} Jul 17 21:28:08.703144 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:08.703095 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" podStartSLOduration=1.636508292 podStartE2EDuration="2.703079201s" totalImagesPullingTime="1.066570909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:28:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:28:08.701869022 +0000 UTC m=+669.471217000" watchObservedRunningTime="2026-07-17 21:28:08.703079201 +0000 UTC m=+669.472427182" Jul 17 21:28:11.229293 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:11.229260 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:28:12.952928 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:12.952892 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:28:16.041516 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:16.041479 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:28:17.260243 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.260205 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:17.260701 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.260269 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:17.272804 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.272778 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:17.743603 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.743574 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlkpn" Jul 17 21:28:17.943228 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.943198 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9"] Jul 17 21:28:17.946898 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.946878 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:17.949751 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.949733 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 17 21:28:17.957281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.957261 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9"] Jul 17 21:28:17.963294 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.963271 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:28:17.977746 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.977719 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:17.977872 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.977761 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpjpg\" (UniqueName: \"kubernetes.io/projected/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-kube-api-access-bpjpg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:17.977872 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.977825 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:17.977872 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.977867 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:17.978024 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.977892 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:17.978024 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:17.977936 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.078748 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.078714 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.078748 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.078761 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.078987 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.078784 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.078987 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.078822 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.078987 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.078891 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.078987 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.078917 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpjpg\" (UniqueName: \"kubernetes.io/projected/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-kube-api-access-bpjpg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.079181 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.079158 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.079536 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.079509 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.079824 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.079798 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.081994 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.081969 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.082188 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.082169 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.092132 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.092111 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpjpg\" (UniqueName: \"kubernetes.io/projected/72e3b4a6-08ae-4936-ad1d-a4e8afa67de9-kube-api-access-bpjpg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xjlc9\" (UID: \"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.257866 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.257836 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:18.393120 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.393088 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9"] Jul 17 21:28:18.395230 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:28:18.395200 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e3b4a6_08ae_4936_ad1d_a4e8afa67de9.slice/crio-da1882e5cdd7bac1d6da1bcdca64a5fad84554db08ac212d4f8ed5fa2d9ccbaf WatchSource:0}: Error finding container da1882e5cdd7bac1d6da1bcdca64a5fad84554db08ac212d4f8ed5fa2d9ccbaf: Status 404 returned error can't find the container with id da1882e5cdd7bac1d6da1bcdca64a5fad84554db08ac212d4f8ed5fa2d9ccbaf Jul 17 21:28:18.396636 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.396616 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 21:28:18.738349 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.738310 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" event={"ID":"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9","Type":"ContainerStarted","Data":"56770db73ea240067303a34fc349ebaffc998fb6db9b258b60c5cedfc569da21"} Jul 17 21:28:18.738566 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.738366 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" event={"ID":"72e3b4a6-08ae-4936-ad1d-a4e8afa67de9","Type":"ContainerStarted","Data":"da1882e5cdd7bac1d6da1bcdca64a5fad84554db08ac212d4f8ed5fa2d9ccbaf"} Jul 17 21:28:18.757632 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:18.757585 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" podStartSLOduration=1.572593682 podStartE2EDuration="1.757570482s" totalImagesPullingTime="184.9768ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:28:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:28:18.756863099 +0000 UTC m=+679.526211084" watchObservedRunningTime="2026-07-17 21:28:18.757570482 +0000 UTC m=+679.526918460" Jul 17 21:28:21.443709 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.443675 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:28:21.620669 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.620642 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b"] Jul 17 21:28:21.623788 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.623768 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.626654 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.626627 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 17 21:28:21.633831 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.633810 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b"] Jul 17 21:28:21.654920 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.654892 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:28:21.713647 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.713524 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.713647 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.713613 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.713814 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.713661 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.713814 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.713733 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ljk9\" (UniqueName: \"kubernetes.io/projected/25c88dd9-90d6-4e3a-b06a-6142537d3c39-kube-api-access-2ljk9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.713814 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.713803 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.713935 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.713826 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25c88dd9-90d6-4e3a-b06a-6142537d3c39-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.817808 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.816736 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.817808 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.816792 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25c88dd9-90d6-4e3a-b06a-6142537d3c39-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.817808 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.816885 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.817808 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.816916 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.817808 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.816945 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.817808 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.816993 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ljk9\" (UniqueName: \"kubernetes.io/projected/25c88dd9-90d6-4e3a-b06a-6142537d3c39-kube-api-access-2ljk9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.822565 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.819101 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.822565 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.819350 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.822565 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.819667 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.825965 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.825943 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25c88dd9-90d6-4e3a-b06a-6142537d3c39-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.826138 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.826096 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25c88dd9-90d6-4e3a-b06a-6142537d3c39-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.827328 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.827308 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ljk9\" (UniqueName: \"kubernetes.io/projected/25c88dd9-90d6-4e3a-b06a-6142537d3c39-kube-api-access-2ljk9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b\" (UID: \"25c88dd9-90d6-4e3a-b06a-6142537d3c39\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:21.933962 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:21.933928 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:22.079864 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:22.079831 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b"] Jul 17 21:28:22.081692 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:28:22.081667 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25c88dd9_90d6_4e3a_b06a_6142537d3c39.slice/crio-c5a7596da192bdbef919d67b74976794aad8a08595522bbf27b891f1ff46155b WatchSource:0}: Error finding container c5a7596da192bdbef919d67b74976794aad8a08595522bbf27b891f1ff46155b: Status 404 returned error can't find the container with id c5a7596da192bdbef919d67b74976794aad8a08595522bbf27b891f1ff46155b Jul 17 21:28:22.757766 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:22.757732 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" event={"ID":"25c88dd9-90d6-4e3a-b06a-6142537d3c39","Type":"ContainerStarted","Data":"a08269defa88106e701bf2655c1f27ab25708b5d91b143bf6be5dcba84b95413"} Jul 17 21:28:22.757766 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:22.757772 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" event={"ID":"25c88dd9-90d6-4e3a-b06a-6142537d3c39","Type":"ContainerStarted","Data":"c5a7596da192bdbef919d67b74976794aad8a08595522bbf27b891f1ff46155b"} Jul 17 21:28:22.776812 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:22.776766 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" podStartSLOduration=1.592451868 podStartE2EDuration="1.776749903s" totalImagesPullingTime="184.298035ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:28:22.773924937 +0000 UTC m=+683.543272917" watchObservedRunningTime="2026-07-17 21:28:22.776749903 +0000 UTC m=+683.546097880" Jul 17 21:28:28.258952 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:28.258918 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:28.259386 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:28.258977 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:28.271436 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:28.271412 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:28.797533 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:28.797505 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xjlc9" Jul 17 21:28:30.581902 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:30.581861 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:28:31.934575 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:31.934527 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:31.934992 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:31.934594 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:31.947416 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:31.947393 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:32.817764 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:32.817730 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b" Jul 17 21:28:57.669969 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:28:57.669932 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:29:22.717048 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:29:22.716994 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:29:22.791324 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:29:22.791293 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:29:23.798472 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:29:23.798436 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:29:23.909091 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:29:23.909061 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:29:50.637105 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:29:50.637063 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:30:00.536697 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:30:00.536660 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:30:11.142859 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:30:11.142823 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:30:29.170291 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:30:29.170212 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:30:36.349949 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:30:36.349915 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:31:26.422182 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:31:26.422148 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:31:27.654647 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:31:27.654604 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:31:51.537130 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:31:51.537034 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:31:59.628946 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:31:59.628921 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:31:59.633335 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:31:59.633316 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:32:05.545272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:05.545238 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:32:08.051234 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:08.051203 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:32:08.152091 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:08.152050 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:32:08.274956 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:08.274924 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:32:13.863801 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:13.863762 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:32:13.966466 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:13.966437 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:32:14.064000 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:14.063974 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:32:14.200970 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:14.200886 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:32:20.063280 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:20.063245 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:32:20.162482 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:20.162444 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:32:36.537610 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:36.537577 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:32:46.233956 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:46.233922 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:32:54.639903 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:32:54.639871 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:33:03.736517 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:33:03.736486 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:33:07.066413 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:33:07.066378 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:33:08.207978 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:33:08.207946 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:33:14.850895 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:33:14.850865 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:33:24.141083 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:33:24.140993 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:33:35.252339 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:33:35.252294 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:33:44.838956 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:33:44.838923 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:33:45.715865 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:33:45.715827 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:34:07.440925 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:34:07.440884 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:34:16.337015 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:34:16.336975 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:34:40.547234 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:34:40.547199 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:34:50.631116 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:34:50.631026 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:34:58.982025 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:34:58.981992 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:35:07.837162 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:35:07.837128 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:35:16.745330 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:35:16.745295 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:35:25.750409 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:35:25.750377 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:35:35.047376 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:35:35.047337 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:35:44.328657 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:35:44.328621 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:36:10.222193 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:10.222157 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n"] Jul 17 21:36:10.222707 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:10.222346 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" podUID="31318d4d-e596-4614-9c2f-331bcc8e00f4" containerName="manager" containerID="cri-o://9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526" gracePeriod=10 Jul 17 21:36:12.683247 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:12.683216 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:36:12.863274 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:12.863236 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31318d4d-e596-4614-9c2f-331bcc8e00f4-extensions-socket-volume\") pod \"31318d4d-e596-4614-9c2f-331bcc8e00f4\" (UID: \"31318d4d-e596-4614-9c2f-331bcc8e00f4\") " Jul 17 21:36:12.863274 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:12.863278 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thctk\" (UniqueName: \"kubernetes.io/projected/31318d4d-e596-4614-9c2f-331bcc8e00f4-kube-api-access-thctk\") pod \"31318d4d-e596-4614-9c2f-331bcc8e00f4\" (UID: \"31318d4d-e596-4614-9c2f-331bcc8e00f4\") " Jul 17 21:36:12.863705 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:12.863678 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31318d4d-e596-4614-9c2f-331bcc8e00f4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "31318d4d-e596-4614-9c2f-331bcc8e00f4" (UID: "31318d4d-e596-4614-9c2f-331bcc8e00f4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 21:36:12.865568 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:12.865530 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31318d4d-e596-4614-9c2f-331bcc8e00f4-kube-api-access-thctk" (OuterVolumeSpecName: "kube-api-access-thctk") pod "31318d4d-e596-4614-9c2f-331bcc8e00f4" (UID: "31318d4d-e596-4614-9c2f-331bcc8e00f4"). InnerVolumeSpecName "kube-api-access-thctk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 21:36:12.964927 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:12.964888 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thctk\" (UniqueName: \"kubernetes.io/projected/31318d4d-e596-4614-9c2f-331bcc8e00f4-kube-api-access-thctk\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:36:12.964927 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:12.964917 2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31318d4d-e596-4614-9c2f-331bcc8e00f4-extensions-socket-volume\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 21:36:13.038960 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:13.038926 2554 generic.go:356] "Generic (PLEG): container finished" podID="31318d4d-e596-4614-9c2f-331bcc8e00f4" containerID="9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526" exitCode=0 Jul 17 21:36:13.039116 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:13.038969 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" Jul 17 21:36:13.039116 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:13.039011 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" event={"ID":"31318d4d-e596-4614-9c2f-331bcc8e00f4","Type":"ContainerDied","Data":"9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526"} Jul 17 21:36:13.039116 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:13.039056 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n" event={"ID":"31318d4d-e596-4614-9c2f-331bcc8e00f4","Type":"ContainerDied","Data":"7f37acb6a33660d90db9e2d752acf768f919d2a836825e60829c270e25e53a32"} Jul 17 21:36:13.039116 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:13.039078 2554 scope.go:117] "RemoveContainer" containerID="9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526" Jul 17 21:36:13.048768 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:13.048753 2554 scope.go:117] "RemoveContainer" containerID="9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526" Jul 17 21:36:13.048999 ip-10-0-131-188 kubenswrapper[2554]: E0717 21:36:13.048980 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526\": container with ID starting with 9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526 not found: ID does not exist" containerID="9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526" Jul 17 21:36:13.049037 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:13.049014 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526"} err="failed to get container status \"9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526\": rpc error: code = NotFound desc = could not find container \"9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526\": container with ID starting with 9b7eb53a595fa8965e7744e000d34b8ba6fbc72b03eb6773daf62f047bda2526 not found: ID does not exist" Jul 17 21:36:13.060968 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:13.060944 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n"] Jul 17 21:36:13.065798 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:13.065777 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fw5n"] Jul 17 21:36:13.719853 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:13.719823 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31318d4d-e596-4614-9c2f-331bcc8e00f4" path="/var/lib/kubelet/pods/31318d4d-e596-4614-9c2f-331bcc8e00f4/volumes" Jul 17 21:36:55.873117 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:55.873085 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:36:59.664962 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:59.664916 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:36:59.666259 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:36:59.666240 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:37:11.337821 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.337788 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q"] Jul 17 21:37:11.338183 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.338150 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31318d4d-e596-4614-9c2f-331bcc8e00f4" containerName="manager" Jul 17 21:37:11.338183 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.338160 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="31318d4d-e596-4614-9c2f-331bcc8e00f4" containerName="manager" Jul 17 21:37:11.338256 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.338215 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="31318d4d-e596-4614-9c2f-331bcc8e00f4" containerName="manager" Jul 17 21:37:11.342499 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.342480 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" Jul 17 21:37:11.345311 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.345291 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-phgnj\"" Jul 17 21:37:11.358895 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.358873 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q"] Jul 17 21:37:11.384012 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.383977 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9kwh\" (UniqueName: \"kubernetes.io/projected/c1b01572-4ec7-4607-9aa4-b894b1de2277-kube-api-access-t9kwh\") pod \"kuadrant-operator-controller-manager-86878c769f-2x66q\" (UID: \"c1b01572-4ec7-4607-9aa4-b894b1de2277\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" Jul 17 21:37:11.384127 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.384108 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1b01572-4ec7-4607-9aa4-b894b1de2277-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2x66q\" (UID: \"c1b01572-4ec7-4607-9aa4-b894b1de2277\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" Jul 17 21:37:11.485041 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.485003 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1b01572-4ec7-4607-9aa4-b894b1de2277-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2x66q\" (UID: \"c1b01572-4ec7-4607-9aa4-b894b1de2277\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" Jul 17 21:37:11.485255 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.485061 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9kwh\" (UniqueName: \"kubernetes.io/projected/c1b01572-4ec7-4607-9aa4-b894b1de2277-kube-api-access-t9kwh\") pod \"kuadrant-operator-controller-manager-86878c769f-2x66q\" (UID: \"c1b01572-4ec7-4607-9aa4-b894b1de2277\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" Jul 17 21:37:11.485466 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.485447 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1b01572-4ec7-4607-9aa4-b894b1de2277-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2x66q\" (UID: \"c1b01572-4ec7-4607-9aa4-b894b1de2277\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" Jul 17 21:37:11.494155 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.494132 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9kwh\" (UniqueName: \"kubernetes.io/projected/c1b01572-4ec7-4607-9aa4-b894b1de2277-kube-api-access-t9kwh\") pod \"kuadrant-operator-controller-manager-86878c769f-2x66q\" (UID: \"c1b01572-4ec7-4607-9aa4-b894b1de2277\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" Jul 17 21:37:11.653205 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.653136 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" Jul 17 21:37:11.793401 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:11.793379 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q"] Jul 17 21:37:11.795636 ip-10-0-131-188 kubenswrapper[2554]: W0717 21:37:11.795608 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b01572_4ec7_4607_9aa4_b894b1de2277.slice/crio-3d03958a4895b0146f0db9cb19178e336d4ffdf3acb47c63aed4c2b164553d57 WatchSource:0}: Error finding container 3d03958a4895b0146f0db9cb19178e336d4ffdf3acb47c63aed4c2b164553d57: Status 404 returned error can't find the container with id 3d03958a4895b0146f0db9cb19178e336d4ffdf3acb47c63aed4c2b164553d57 Jul 17 21:37:12.323879 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:12.323838 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" event={"ID":"c1b01572-4ec7-4607-9aa4-b894b1de2277","Type":"ContainerStarted","Data":"ef0d797da923344a2530a0e6dfdae63a3a4c94042e45db0df3a2a3b29bc8b388"} Jul 17 21:37:12.324050 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:12.323901 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" event={"ID":"c1b01572-4ec7-4607-9aa4-b894b1de2277","Type":"ContainerStarted","Data":"3d03958a4895b0146f0db9cb19178e336d4ffdf3acb47c63aed4c2b164553d57"} Jul 17 21:37:12.324104 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:12.324051 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" Jul 17 21:37:12.346415 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:12.346366 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" podStartSLOduration=1.346351851 podStartE2EDuration="1.346351851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 21:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 21:37:12.343378388 +0000 UTC m=+1213.112726368" watchObservedRunningTime="2026-07-17 21:37:12.346351851 +0000 UTC m=+1213.115699828" Jul 17 21:37:23.331259 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:23.331232 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2x66q" Jul 17 21:37:34.594928 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:34.594898 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:37:43.961046 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:43.961010 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:37:56.554227 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:37:56.554192 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:38:09.894258 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:38:09.894226 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:38:20.649781 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:38:20.649740 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:38:29.944189 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:38:29.944157 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:38:38.836916 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:38:38.836880 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:38:49.438161 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:38:49.438120 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:39:00.051977 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:39:00.051933 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:39:07.638016 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:39:07.637986 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:39:16.645716 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:39:16.645678 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:39:25.953776 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:39:25.953690 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:39:39.038449 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:39:39.038409 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:39:43.438434 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:39:43.438401 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:39:54.246036 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:39:54.246002 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:40:03.742187 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:40:03.742159 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:40:13.438347 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:40:13.438312 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:40:23.835867 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:40:23.835830 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:40:36.052224 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:40:36.052187 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:40:44.851065 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:40:44.851031 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:40:54.049614 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:40:54.049508 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:41:01.837876 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:41:01.837842 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:41:12.647325 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:41:12.647289 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:41:22.044933 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:41:22.044902 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:41:35.346713 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:41:35.346677 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:41:44.539246 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:41:44.539211 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:41:54.647945 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:41:54.647906 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:41:59.695456 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:41:59.695429 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:41:59.695903 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:41:59.695692 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:42:05.451678 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:42:05.451645 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:42:16.843899 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:42:16.843859 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:42:26.144036 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:42:26.143958 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:42:38.847469 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:42:38.847434 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:42:51.246711 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:42:51.246677 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:43:02.240168 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:02.240134 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:43:04.237158 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:04.237126 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:10.858170 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:10.858129 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:14.238901 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:14.238860 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:43:19.109717 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:19.109685 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:23.440115 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:23.440083 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:43:24.189901 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:24.189874 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:27.049986 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:27.049937 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:27.148120 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:27.148086 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:27.488703 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:27.488608 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:32.932255 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:32.932222 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:43:33.435859 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:33.435824 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:33.536372 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:33.536343 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:33.648237 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:33.648201 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:33.714945 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:33.714872 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:34.834045 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:34.834010 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:34.927099 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:34.927065 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:35.062709 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:35.062675 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:35.331125 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:35.331095 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:35.517344 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:35.517316 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:35.618484 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:35.618396 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:37.680674 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:37.680646 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:43:44.546318 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:43:44.546286 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:44:19.964204 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:44:19.964171 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:45:20.545142 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:45:20.545053 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:39.895603 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:39.895570 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:40.248191 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:40.248103 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:42.641360 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:42.641328 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:46:45.261137 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:45.261110 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:46.289060 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:46.289033 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:49.801162 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:49.801128 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:49.903087 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:49.903052 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:51.494721 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:51.494697 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:51.894767 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:51.894741 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:51.996092 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:51.996063 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:52.181465 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:52.181397 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:52.280913 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:52.280882 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:52.359234 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:52.359204 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:53.584701 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:53.584670 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:53.697811 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:53.697765 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:53.965939 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:53.965820 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:54.215007 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:54.214977 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:54.315783 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:54.315749 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:55.960694 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:55.960668 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:46:59.725698 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:59.725669 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:46:59.728113 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:46:59.728066 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:47:01.562623 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:47:01.562594 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:47:58.226224 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:47:58.226191 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:47:58.327661 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:47:58.327617 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:01.162212 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:01.162179 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:48:03.969315 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:03.969272 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:04.565786 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:04.565747 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:09.751644 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:09.751595 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:10.760198 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:10.760171 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:10.861141 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:10.861103 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:10.995059 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:10.995030 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:11.091657 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:11.091624 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:12.268690 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:12.268657 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:12.378015 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:12.377972 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:12.470770 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:12.470741 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:12.952191 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:12.952158 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:13.059147 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:13.059110 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:13.162351 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:13.162324 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:14.221945 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:14.221912 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:48:14.749805 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:48:14.749776 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:49:25.348725 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:49:25.348689 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:49:25.407711 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:49:25.407683 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:51:50.715804 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:51:50.715772 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:51:51.955073 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:51:51.955047 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:51:57.982681 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:51:57.982650 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:51:58.082817 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:51:58.082787 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:51:59.753802 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:51:59.753778 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:51:59.758028 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:51:59.757999 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:52:13.365005 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:52:13.364970 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:53:04.554781 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:53:04.554734 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:53:04.606920 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:53:04.606893 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:17.411245 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:17.411212 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:17.519523 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:17.519489 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:20.542940 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:20.542851 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:54:22.600471 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:22.600429 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:23.651781 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:23.651748 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:29.298845 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:29.298800 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:29.359097 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:29.359066 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:29.565632 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:29.565563 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:30.673442 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:30.673323 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:30.754599 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:30.754498 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:30.831188 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:30.831163 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:31.057561 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:31.057513 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:31.357989 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:31.357901 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:31.458976 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:31.458945 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:34.341933 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:34.341903 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:54:53.969755 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:53.969721 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:54.071665 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:54.071638 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:54:57.909431 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:54:57.909404 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:55:52.929325 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:55:52.929250 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:56:04.183267 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:56:04.183233 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:56:51.466976 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:56:51.466940 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:56:51.563400 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:56:51.563362 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:56:54.882377 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:56:54.882342 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:56:56.018965 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:56:56.018937 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:56:59.787321 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:56:59.787291 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:56:59.793281 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:56:59.793258 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 21:57:35.227984 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:35.227891 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:35.381184 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:35.381144 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:38.042528 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:38.042488 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:57:40.716686 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:40.716637 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:41.369384 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:41.369351 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:47.186451 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:47.186418 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:47.286120 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:47.286088 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:47.669566 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:47.669521 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:47.769068 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:47.769035 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:47.866895 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:47.866866 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:49.254311 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:49.254281 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:55.098280 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:55.098235 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:55.206497 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:55.206467 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:55.299290 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:55.299260 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:55.400150 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:55.400017 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:55.774272 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:55.774241 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:57:56.964706 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:57:56.964654 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:58:21.751842 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:58:21.751809 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:58:27.554016 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:58:27.553990 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:58:27.666457 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:58:27.666422 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:58:27.762477 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:58:27.762446 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:58:43.614331 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:58:43.614299 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:58:49.395101 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:58:49.395065 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:58:49.503531 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:58:49.503481 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:58:49.600890 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:58:49.600861 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:00.101478 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:00.101442 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:00.883216 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:00.883185 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:06.845535 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:06.845501 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:06.916532 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:06.916501 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:06.990657 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:06.990622 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:07.091355 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:07.091324 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:07.864663 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:07.864630 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:07.964454 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:07.964421 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:08.064620 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:08.064583 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:09.340896 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:09.340862 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:59:09.635640 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:09.635563 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:15.868837 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:15.868787 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:15.934961 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:15.934934 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:16.543716 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:16.543681 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:16.942653 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:16.942573 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:59:22.605277 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:22.605225 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:22.688533 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:22.688506 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:22.838288 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:22.838251 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:23.167505 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:23.167477 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:23.705201 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:23.705171 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:25.446847 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:25.446816 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:59:26.105086 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:26.105053 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:26.205513 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:26.205468 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:26.445513 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:26.445435 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:26.773700 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:26.773666 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:27.307625 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:27.307592 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:27.411005 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:27.410975 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:30.542800 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:30.542769 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:31.187377 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:31.187345 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:31.262943 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:31.262912 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:31.603646 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:31.603610 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:35.853407 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:35.853374 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:59:37.368269 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:37.368236 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:37.437942 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:37.437909 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:48.275537 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:48.275499 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:51.170715 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:51.170682 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 21:59:52.890148 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:52.890121 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:58.918153 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:58.918124 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 21:59:59.005646 ip-10-0-131-188 kubenswrapper[2554]: I0717 21:59:59.005618 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:03.562143 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:03.562107 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:03.793913 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:03.793883 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:03.917617 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:03.917527 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:04.011538 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:04.011507 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:04.128273 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:04.128239 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:07.258758 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:07.258721 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:09.979700 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:09.979665 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:10.119208 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:10.119179 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:10.241615 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:10.241506 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:10.393915 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:10.393888 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:16.080728 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:16.080694 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:16.139012 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:16.138985 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:16.740690 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:16.740659 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:22.826603 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:22.826519 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:23.145820 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:23.145739 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:23.245781 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:23.245752 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:26.535868 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:26.535834 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:26.708002 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:26.707971 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:26.833794 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:26.833715 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:27.285324 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:27.285286 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:27.385799 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:27.385764 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:30.947622 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:30.947588 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:33.137413 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:33.137378 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:00:51.240497 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.240464 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn"] Jul 17 22:00:51.244061 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.244045 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.247934 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.247905 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-479788cb\"/\"e2e-lifecycle-model-479788cb-kserve-self-signed-certs\"" Jul 17 22:00:51.248055 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.247936 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-479788cb\"/\"openshift-service-ca.crt\"" Jul 17 22:00:51.248055 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.247989 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-479788cb\"/\"kube-root-ca.crt\"" Jul 17 22:00:51.248143 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.248055 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-479788cb\"/\"default-dockercfg-kjl5z\"" Jul 17 22:00:51.252105 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.252081 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn"] Jul 17 22:00:51.340747 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.340713 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:00:51.381900 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.381852 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-model-cache\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.381900 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.381893 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-dshm\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.382126 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.381936 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tls-certs\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.382126 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.381981 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wktp\" (UniqueName: \"kubernetes.io/projected/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-kube-api-access-8wktp\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.382126 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.382048 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tmp-dir\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.382126 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.382082 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-home\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.483119 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.483086 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tmp-dir\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.483119 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.483124 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-home\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.483383 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.483151 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-model-cache\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.483383 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.483259 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-dshm\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.483383 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.483316 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tls-certs\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.483539 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.483396 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wktp\" (UniqueName: \"kubernetes.io/projected/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-kube-api-access-8wktp\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.483624 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.483570 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tmp-dir\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.483624 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.483598 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-home\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.483751 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.483730 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-model-cache\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.485843 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.485816 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-dshm\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.486020 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.486004 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tls-certs\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.492559 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.492497 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wktp\" (UniqueName: \"kubernetes.io/projected/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-kube-api-access-8wktp\") pod \"e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.553811 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.553779 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:00:51.695160 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:51.694899 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn"] Jul 17 22:00:51.698018 ip-10-0-131-188 kubenswrapper[2554]: W0717 22:00:51.697981 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3acdada_94f6_42a6_8b1d_cde8e74e3b5d.slice/crio-02d92e601a48778e43530fa94db98d59452474e9df5cefeff62b4e2e76568236 WatchSource:0}: Error finding container 02d92e601a48778e43530fa94db98d59452474e9df5cefeff62b4e2e76568236: Status 404 returned error can't find the container with id 02d92e601a48778e43530fa94db98d59452474e9df5cefeff62b4e2e76568236 Jul 17 22:00:52.093266 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:52.093233 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" event={"ID":"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d","Type":"ContainerStarted","Data":"e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202"} Jul 17 22:00:52.093429 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:52.093276 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" event={"ID":"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d","Type":"ContainerStarted","Data":"02d92e601a48778e43530fa94db98d59452474e9df5cefeff62b4e2e76568236"} Jul 17 22:00:52.111786 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:52.111736 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" podStartSLOduration=1.111721834 podStartE2EDuration="1.111721834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 22:00:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 22:00:52.110279441 +0000 UTC m=+2632.879627421" watchObservedRunningTime="2026-07-17 22:00:52.111721834 +0000 UTC m=+2632.881069811" Jul 17 22:00:54.135835 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:00:54.135800 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:01:01.553961 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:01.553914 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:01:01.554422 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:01.553977 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:01:01.566985 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:01.566957 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:01:02.157285 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:02.157252 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:01:16.244334 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:16.244304 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:01:29.839531 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:29.839478 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:01:33.445125 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:33.445094 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:01:34.410263 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:34.410230 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:01:39.387889 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:39.387857 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn"] Jul 17 22:01:39.483053 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:39.483019 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:01:52.341082 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:52.340997 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:01:52.438024 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:52.437986 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:01:54.408739 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.408695 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" podUID="d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" containerName="main" containerID="cri-o://e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202" gracePeriod=45 Jul 17 22:01:54.661986 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.661928 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:01:54.760533 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.760499 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-model-cache\") pod \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " Jul 17 22:01:54.760741 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.760586 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wktp\" (UniqueName: \"kubernetes.io/projected/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-kube-api-access-8wktp\") pod \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " Jul 17 22:01:54.760741 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.760621 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-dshm\") pod \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " Jul 17 22:01:54.760741 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.760637 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tmp-dir\") pod \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " Jul 17 22:01:54.760741 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.760655 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tls-certs\") pod \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " Jul 17 22:01:54.760741 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.760687 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-home\") pod \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\" (UID: \"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d\") " Jul 17 22:01:54.761002 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.760817 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-model-cache" (OuterVolumeSpecName: "model-cache") pod "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" (UID: "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:01:54.761002 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.760883 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" (UID: "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:01:54.761002 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.760966 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-home" (OuterVolumeSpecName: "home") pod "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" (UID: "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:01:54.761188 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.761108 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tmp-dir\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:01:54.761188 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.761127 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-home\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:01:54.761188 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.761140 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-model-cache\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:01:54.763024 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.763000 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-kube-api-access-8wktp" (OuterVolumeSpecName: "kube-api-access-8wktp") pod "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" (UID: "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d"). InnerVolumeSpecName "kube-api-access-8wktp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 22:01:54.763418 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.763398 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-dshm" (OuterVolumeSpecName: "dshm") pod "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" (UID: "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:01:54.763511 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.763416 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" (UID: "d3acdada-94f6-42a6-8b1d-cde8e74e3b5d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 22:01:54.862197 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.862156 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wktp\" (UniqueName: \"kubernetes.io/projected/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-kube-api-access-8wktp\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:01:54.862197 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.862189 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-dshm\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:01:54.862197 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:54.862203 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d-tls-certs\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:01:55.402273 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:55.402243 2554 generic.go:356] "Generic (PLEG): container finished" podID="d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" containerID="e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202" exitCode=0 Jul 17 22:01:55.402454 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:55.402292 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" Jul 17 22:01:55.402454 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:55.402336 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" event={"ID":"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d","Type":"ContainerDied","Data":"e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202"} Jul 17 22:01:55.402454 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:55.402379 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn" event={"ID":"d3acdada-94f6-42a6-8b1d-cde8e74e3b5d","Type":"ContainerDied","Data":"02d92e601a48778e43530fa94db98d59452474e9df5cefeff62b4e2e76568236"} Jul 17 22:01:55.402454 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:55.402397 2554 scope.go:117] "RemoveContainer" containerID="e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202" Jul 17 22:01:55.412227 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:55.412041 2554 scope.go:117] "RemoveContainer" containerID="e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202" Jul 17 22:01:55.412445 ip-10-0-131-188 kubenswrapper[2554]: E0717 22:01:55.412308 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202\": container with ID starting with e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202 not found: ID does not exist" containerID="e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202" Jul 17 22:01:55.412445 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:55.412331 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202"} err="failed to get container status \"e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202\": rpc error: code = NotFound desc = could not find container \"e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202\": container with ID starting with e3ab9eaafea6e146ad09695a53d2d98b687494141a33697714e066e6ebf57202 not found: ID does not exist" Jul 17 22:01:55.432907 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:55.432883 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn"] Jul 17 22:01:55.441187 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:55.441164 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-479788cb/e2e-lifecycle-model-479788cb-kserve-59866c55d7-gr8qn"] Jul 17 22:01:55.720016 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:55.719936 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" path="/var/lib/kubelet/pods/d3acdada-94f6-42a6-8b1d-cde8e74e3b5d/volumes" Jul 17 22:01:59.819909 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:59.819870 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 22:01:59.825188 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:01:59.825167 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 22:02:15.847609 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:15.847565 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:17.185459 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:17.185422 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:22.632083 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:22.632051 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:22.702036 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:22.702010 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:26.544835 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:26.544803 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:26.758440 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:26.758408 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:27.767062 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:27.767013 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:27.868925 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:27.868890 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:27.967521 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:27.967480 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:28.654465 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:28.654427 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:28.755709 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:28.755683 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:28.855423 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:28.855384 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:30.206524 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:30.206492 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:30.940396 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:30.940362 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:02:36.392966 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:36.392937 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:36.469960 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:36.469926 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:37.034359 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:37.034331 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:42.801775 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:42.801728 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:42.849665 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:42.849631 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:02:42.911350 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:42.911313 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:43.003923 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:43.003888 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:53.507971 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:53.507919 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:02:53.848237 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:02:53.848207 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:14.784140 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:14.784103 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:20.485415 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:20.485330 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:20.585402 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:20.585368 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:20.824430 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:20.824401 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:21.108728 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:21.108647 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:21.501948 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:21.501856 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:23.576181 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:23.576138 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:23.673018 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:23.672984 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:23.860823 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:23.860734 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:24.368848 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:24.368781 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:24.504642 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:24.504607 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:28.059557 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:28.059514 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:28.385617 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:28.385515 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:28.687447 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:28.687367 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:28.850944 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:28.850908 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:31.442876 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:31.442842 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:03:31.728048 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:31.727969 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:31.828856 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:31.828792 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:31.990410 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:31.990300 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:32.198479 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:32.198452 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:32.613072 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:32.613024 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:32.715367 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:32.715331 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:36.118232 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:36.118191 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:42.631502 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:42.631458 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:03:45.170705 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:45.170663 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:45.271373 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:45.271343 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:52.125135 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.125101 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75"] Jul 17 22:03:52.125724 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.125699 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" containerName="main" Jul 17 22:03:52.125724 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.125717 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" containerName="main" Jul 17 22:03:52.125863 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.125806 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3acdada-94f6-42a6-8b1d-cde8e74e3b5d" containerName="main" Jul 17 22:03:52.129089 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.129069 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.133019 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.132998 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-592f58\"/\"openshift-service-ca.crt\"" Jul 17 22:03:52.134239 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.134220 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-592f58\"/\"kube-root-ca.crt\"" Jul 17 22:03:52.135192 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.135172 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-592f58\"/\"test-model-592f58-kserve-self-signed-certs\"" Jul 17 22:03:52.135344 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.135304 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-592f58\"/\"default-dockercfg-x8fmx\"" Jul 17 22:03:52.147660 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.147631 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75"] Jul 17 22:03:52.184342 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.184306 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:52.197914 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.197884 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-model-cache\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.197914 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.197911 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tls-certs\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.198077 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.198025 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-home\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.198077 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.198048 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tqrv\" (UniqueName: \"kubernetes.io/projected/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-kube-api-access-8tqrv\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.198157 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.198090 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-dshm\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.198157 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.198114 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tmp-dir\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.298844 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.298809 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-home\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.298844 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.298847 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tqrv\" (UniqueName: \"kubernetes.io/projected/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-kube-api-access-8tqrv\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.299074 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.298883 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-dshm\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.299074 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.298905 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tmp-dir\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.299074 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.298937 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-model-cache\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.299074 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.298960 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tls-certs\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.299326 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.299303 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-home\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.299368 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.299328 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-model-cache\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.299434 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.299374 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tmp-dir\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.301351 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.301330 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-dshm\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.301515 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.301498 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tls-certs\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.313559 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.313503 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tqrv\" (UniqueName: \"kubernetes.io/projected/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-kube-api-access-8tqrv\") pod \"test-model-592f58-kserve-6666656df5-jks75\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.438279 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.438190 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:03:52.586861 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.586834 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75"] Jul 17 22:03:52.588798 ip-10-0-131-188 kubenswrapper[2554]: W0717 22:03:52.588770 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59ce4675_257e_45ac_b0cd_aa2f07dd67ef.slice/crio-fe4b8cb8ec3cbb5c4ed5d9de639851adef3a296b58de78bee41d877f2d937bbf WatchSource:0}: Error finding container fe4b8cb8ec3cbb5c4ed5d9de639851adef3a296b58de78bee41d877f2d937bbf: Status 404 returned error can't find the container with id fe4b8cb8ec3cbb5c4ed5d9de639851adef3a296b58de78bee41d877f2d937bbf Jul 17 22:03:52.967320 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.967281 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" event={"ID":"59ce4675-257e-45ac-b0cd-aa2f07dd67ef","Type":"ContainerStarted","Data":"0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb"} Jul 17 22:03:52.967320 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.967322 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" event={"ID":"59ce4675-257e-45ac-b0cd-aa2f07dd67ef","Type":"ContainerStarted","Data":"fe4b8cb8ec3cbb5c4ed5d9de639851adef3a296b58de78bee41d877f2d937bbf"} Jul 17 22:03:52.986154 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:52.986102 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" podStartSLOduration=0.986086894 podStartE2EDuration="986.086894ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 22:03:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 22:03:52.984932023 +0000 UTC m=+2813.754280003" watchObservedRunningTime="2026-07-17 22:03:52.986086894 +0000 UTC m=+2813.755434871" Jul 17 22:03:53.622311 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:53.622282 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:53.722026 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:53.721991 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:03:56.052861 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:03:56.052825 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:04:02.438741 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:02.438707 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:04:02.438741 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:02.438749 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:04:02.451696 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:02.451676 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:04:03.028822 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:03.028797 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:04:08.140126 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:08.140091 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:04:09.436261 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.436225 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl"] Jul 17 22:04:09.439953 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.439934 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.445094 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.445065 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6be22b\"/\"default-dockercfg-zm6ww\"" Jul 17 22:04:09.445211 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.445103 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6be22b\"/\"openshift-service-ca.crt\"" Jul 17 22:04:09.445211 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.445137 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6be22b\"/\"kube-root-ca.crt\"" Jul 17 22:04:09.445211 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.445147 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6be22b\"/\"test-model-6be22b-kserve-self-signed-certs\"" Jul 17 22:04:09.456948 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.456920 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl"] Jul 17 22:04:09.479645 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.479613 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:04:09.558580 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.558523 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-model-cache\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.558832 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.558697 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-home\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.558832 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.558731 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tls-certs\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.558832 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.558761 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tmp-dir\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.558832 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.558783 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdczd\" (UniqueName: \"kubernetes.io/projected/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-kube-api-access-mdczd\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.559041 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.558894 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-dshm\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.659653 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.659617 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-dshm\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.659841 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.659696 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-model-cache\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.659841 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.659766 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-home\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.659841 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.659789 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tls-certs\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.659841 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.659815 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tmp-dir\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.659841 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.659837 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdczd\" (UniqueName: \"kubernetes.io/projected/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-kube-api-access-mdczd\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.660179 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.660138 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-model-cache\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.660179 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.660152 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-home\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.660279 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.660208 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tmp-dir\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.662134 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.662113 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-dshm\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.662445 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.662428 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tls-certs\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.669449 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.669432 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdczd\" (UniqueName: \"kubernetes.io/projected/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-kube-api-access-mdczd\") pod \"test-model-6be22b-kserve-866dc4f7c9-6hbvl\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.748638 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.748561 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:09.896298 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:09.896272 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl"] Jul 17 22:04:09.898764 ip-10-0-131-188 kubenswrapper[2554]: W0717 22:04:09.898733 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09ac4bbd_e7ce_4b13_8fe9_84ee37c8a69d.slice/crio-4e2fb7f2e9c8591d3cf1492c8f6ffe75b2319e18cd665282cdf18b607e694310 WatchSource:0}: Error finding container 4e2fb7f2e9c8591d3cf1492c8f6ffe75b2319e18cd665282cdf18b607e694310: Status 404 returned error can't find the container with id 4e2fb7f2e9c8591d3cf1492c8f6ffe75b2319e18cd665282cdf18b607e694310 Jul 17 22:04:10.053890 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:10.053848 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" event={"ID":"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d","Type":"ContainerStarted","Data":"6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9"} Jul 17 22:04:10.053890 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:10.053890 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" event={"ID":"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d","Type":"ContainerStarted","Data":"4e2fb7f2e9c8591d3cf1492c8f6ffe75b2319e18cd665282cdf18b607e694310"} Jul 17 22:04:10.074204 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:10.074158 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" podStartSLOduration=1.074143636 podStartE2EDuration="1.074143636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 22:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 22:04:10.072245944 +0000 UTC m=+2830.841593921" watchObservedRunningTime="2026-07-17 22:04:10.074143636 +0000 UTC m=+2830.843491614" Jul 17 22:04:19.749100 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:19.749067 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:19.749505 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:19.749123 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:19.761772 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:19.761740 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:20.120910 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:20.120877 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:21.138422 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:21.138383 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:04:39.556566 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:39.556506 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75"] Jul 17 22:04:39.582082 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:39.582052 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:04:40.521293 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:40.521259 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl"] Jul 17 22:04:40.541774 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:40.541747 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:04:41.299634 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:41.299604 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:04:41.397805 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:41.397779 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:04:44.445312 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:44.445265 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:04:49.510039 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:49.510006 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:04:49.610449 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:49.610419 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:04:50.449867 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:50.449837 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:04:51.093511 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:51.093480 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:04:53.795338 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:53.795302 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:04:54.577327 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.577282 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" podUID="59ce4675-257e-45ac-b0cd-aa2f07dd67ef" containerName="main" containerID="cri-o://0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb" gracePeriod=45 Jul 17 22:04:54.832777 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.832709 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:04:54.883354 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.883322 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-dshm\") pod \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " Jul 17 22:04:54.883511 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.883396 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-model-cache\") pod \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " Jul 17 22:04:54.883511 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.883417 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tqrv\" (UniqueName: \"kubernetes.io/projected/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-kube-api-access-8tqrv\") pod \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " Jul 17 22:04:54.883511 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.883448 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tmp-dir\") pod \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " Jul 17 22:04:54.883511 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.883466 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-home\") pod \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " Jul 17 22:04:54.883777 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.883525 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tls-certs\") pod \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\" (UID: \"59ce4675-257e-45ac-b0cd-aa2f07dd67ef\") " Jul 17 22:04:54.883777 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.883663 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-model-cache" (OuterVolumeSpecName: "model-cache") pod "59ce4675-257e-45ac-b0cd-aa2f07dd67ef" (UID: "59ce4675-257e-45ac-b0cd-aa2f07dd67ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:04:54.883777 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.883752 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59ce4675-257e-45ac-b0cd-aa2f07dd67ef" (UID: "59ce4675-257e-45ac-b0cd-aa2f07dd67ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:04:54.883927 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.883858 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-home" (OuterVolumeSpecName: "home") pod "59ce4675-257e-45ac-b0cd-aa2f07dd67ef" (UID: "59ce4675-257e-45ac-b0cd-aa2f07dd67ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:04:54.883927 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.883903 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-model-cache\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:54.883927 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.883924 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tmp-dir\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:54.885697 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.885673 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-dshm" (OuterVolumeSpecName: "dshm") pod "59ce4675-257e-45ac-b0cd-aa2f07dd67ef" (UID: "59ce4675-257e-45ac-b0cd-aa2f07dd67ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:04:54.885811 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.885797 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59ce4675-257e-45ac-b0cd-aa2f07dd67ef" (UID: "59ce4675-257e-45ac-b0cd-aa2f07dd67ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 22:04:54.885968 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.885952 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-kube-api-access-8tqrv" (OuterVolumeSpecName: "kube-api-access-8tqrv") pod "59ce4675-257e-45ac-b0cd-aa2f07dd67ef" (UID: "59ce4675-257e-45ac-b0cd-aa2f07dd67ef"). InnerVolumeSpecName "kube-api-access-8tqrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 22:04:54.984598 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.984567 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-home\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:54.984598 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.984590 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-tls-certs\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:54.984598 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.984602 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-dshm\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:54.984857 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:54.984610 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tqrv\" (UniqueName: \"kubernetes.io/projected/59ce4675-257e-45ac-b0cd-aa2f07dd67ef-kube-api-access-8tqrv\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:55.276516 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.276487 2554 generic.go:356] "Generic (PLEG): container finished" podID="59ce4675-257e-45ac-b0cd-aa2f07dd67ef" containerID="0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb" exitCode=0 Jul 17 22:04:55.276694 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.276531 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" Jul 17 22:04:55.276694 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.276574 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" event={"ID":"59ce4675-257e-45ac-b0cd-aa2f07dd67ef","Type":"ContainerDied","Data":"0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb"} Jul 17 22:04:55.276694 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.276617 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75" event={"ID":"59ce4675-257e-45ac-b0cd-aa2f07dd67ef","Type":"ContainerDied","Data":"fe4b8cb8ec3cbb5c4ed5d9de639851adef3a296b58de78bee41d877f2d937bbf"} Jul 17 22:04:55.276694 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.276639 2554 scope.go:117] "RemoveContainer" containerID="0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb" Jul 17 22:04:55.286208 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.286189 2554 scope.go:117] "RemoveContainer" containerID="0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb" Jul 17 22:04:55.286513 ip-10-0-131-188 kubenswrapper[2554]: E0717 22:04:55.286447 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb\": container with ID starting with 0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb not found: ID does not exist" containerID="0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb" Jul 17 22:04:55.286633 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.286526 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb"} err="failed to get container status \"0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb\": rpc error: code = NotFound desc = could not find container \"0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb\": container with ID starting with 0ef7248068f0333310eac80e081bc0ceb464c71e5dd84bfdb621533a29cb05bb not found: ID does not exist" Jul 17 22:04:55.297318 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.297296 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75"] Jul 17 22:04:55.300082 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.300060 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-592f58/test-model-592f58-kserve-6666656df5-jks75"] Jul 17 22:04:55.542514 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.542430 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" podUID="09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" containerName="main" containerID="cri-o://6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9" gracePeriod=45 Jul 17 22:04:55.720950 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.720920 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59ce4675-257e-45ac-b0cd-aa2f07dd67ef" path="/var/lib/kubelet/pods/59ce4675-257e-45ac-b0cd-aa2f07dd67ef/volumes" Jul 17 22:04:55.736393 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.736367 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:04:55.810234 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.810206 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:55.892012 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.891985 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-model-cache\") pod \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " Jul 17 22:04:55.892012 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.892015 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-dshm\") pod \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " Jul 17 22:04:55.892468 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.892063 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-home\") pod \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " Jul 17 22:04:55.892468 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.892103 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tls-certs\") pod \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " Jul 17 22:04:55.892468 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.892131 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tmp-dir\") pod \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " Jul 17 22:04:55.892468 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.892171 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdczd\" (UniqueName: \"kubernetes.io/projected/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-kube-api-access-mdczd\") pod \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\" (UID: \"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d\") " Jul 17 22:04:55.892468 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.892282 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-model-cache" (OuterVolumeSpecName: "model-cache") pod "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" (UID: "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:04:55.892468 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.892358 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" (UID: "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:04:55.892468 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.892387 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-home" (OuterVolumeSpecName: "home") pod "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" (UID: "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:04:55.892468 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.892445 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-model-cache\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:55.892468 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.892458 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tmp-dir\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:55.894378 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.894357 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-dshm" (OuterVolumeSpecName: "dshm") pod "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" (UID: "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:04:55.894485 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.894380 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" (UID: "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 22:04:55.894485 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.894422 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-kube-api-access-mdczd" (OuterVolumeSpecName: "kube-api-access-mdczd") pod "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" (UID: "09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d"). InnerVolumeSpecName "kube-api-access-mdczd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 22:04:55.993721 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.993681 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdczd\" (UniqueName: \"kubernetes.io/projected/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-kube-api-access-mdczd\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:55.993721 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.993717 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-dshm\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:55.993721 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.993725 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-home\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:55.993721 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:55.993734 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d-tls-certs\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:04:56.284295 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:56.284264 2554 generic.go:356] "Generic (PLEG): container finished" podID="09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" containerID="6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9" exitCode=0 Jul 17 22:04:56.284443 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:56.284314 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" Jul 17 22:04:56.284443 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:56.284341 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" event={"ID":"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d","Type":"ContainerDied","Data":"6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9"} Jul 17 22:04:56.284443 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:56.284379 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl" event={"ID":"09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d","Type":"ContainerDied","Data":"4e2fb7f2e9c8591d3cf1492c8f6ffe75b2319e18cd665282cdf18b607e694310"} Jul 17 22:04:56.284443 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:56.284394 2554 scope.go:117] "RemoveContainer" containerID="6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9" Jul 17 22:04:56.299645 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:56.299617 2554 scope.go:117] "RemoveContainer" containerID="6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9" Jul 17 22:04:56.299887 ip-10-0-131-188 kubenswrapper[2554]: E0717 22:04:56.299866 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9\": container with ID starting with 6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9 not found: ID does not exist" containerID="6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9" Jul 17 22:04:56.299929 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:56.299898 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9"} err="failed to get container status \"6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9\": rpc error: code = NotFound desc = could not find container \"6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9\": container with ID starting with 6de790d512626d330bb970993e648d6c37899654e891a4c9fbff9c389b0e37a9 not found: ID does not exist" Jul 17 22:04:56.312583 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:56.312529 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl"] Jul 17 22:04:56.314526 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:56.314506 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-6be22b/test-model-6be22b-kserve-866dc4f7c9-6hbvl"] Jul 17 22:04:57.720529 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:57.720494 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" path="/var/lib/kubelet/pods/09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d/volumes" Jul 17 22:04:58.239333 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:04:58.239299 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:05:01.549034 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:01.549003 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:01.650926 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:01.650892 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:16.399266 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:16.399237 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:26.629810 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:26.629776 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:27.334507 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:27.334475 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:27.434668 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:27.434634 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:27.541186 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:27.541156 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:37.197366 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:37.197313 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:40.335903 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:40.335870 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:05:41.830933 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:41.830902 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:47.790324 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:47.790288 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:47.854498 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:47.854468 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:52.497585 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:52.497531 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:52.765768 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:52.765684 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:53.069606 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:53.069571 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:53.384182 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:53.384107 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:53.589612 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:53.589582 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:56.615195 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:56.615157 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:56.696896 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:56.696858 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:56.832070 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:56.832037 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:56.962342 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:56.962262 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:57.369780 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:57.369752 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:05:57.469320 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:05:57.469292 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:01.092914 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:01.092875 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:01.246251 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:01.246204 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:06:01.554288 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:01.554256 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:01.854273 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:01.854190 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:02.144349 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:02.144259 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:04.672211 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:04.672174 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:04.770187 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:04.770155 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:04.906318 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:04.906288 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:05.384413 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:05.384385 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:05.531393 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:05.531358 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:05.633527 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:05.633492 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:09.141558 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:09.141519 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:13.034016 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:13.033974 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:06:17.769228 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:17.769194 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:17.875516 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:17.875482 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:26.466807 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:26.466724 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:26.570646 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:26.570617 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:27.158112 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:27.158081 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:30.943151 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:30.943116 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:06:32.159311 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:32.159269 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:37.982715 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:37.982678 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:38.084331 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:38.084286 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:42.839601 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:42.839564 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:43.057443 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:43.057414 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:53.078445 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:53.078411 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:53.188089 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:53.188051 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:53.341506 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:53.341426 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:56.446797 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:56.446755 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:06:58.262531 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:58.262495 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:06:59.849026 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:59.849003 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 22:06:59.854684 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:06:59.854662 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 22:07:04.038828 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:04.038789 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:04.105586 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:04.105530 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:08.931085 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:08.931051 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:09.146515 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:09.146482 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:09.457597 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:09.457535 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:09.753290 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:09.753216 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:09.985213 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:09.985179 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:11.006336 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:11.006299 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:13.163275 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:13.163244 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:13.246301 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:13.246268 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:13.317588 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:13.317500 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:13.491668 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:13.491581 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:13.845567 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:13.845518 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:13.946358 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:13.946329 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:15.565446 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:15.565412 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:15.666758 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:15.666726 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:17.442889 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:17.442854 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:07:17.627373 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:17.627339 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:19.210441 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:19.210405 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:21.189920 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:21.189881 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:21.310382 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:21.310349 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:21.409234 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:21.409208 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:21.622119 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:21.622065 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:21.962102 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:21.962016 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:22.090410 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:22.090379 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:25.656464 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:25.656429 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:29.342451 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:29.342412 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:07:33.541495 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.541459 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25"] Jul 17 22:07:33.541893 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.541859 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59ce4675-257e-45ac-b0cd-aa2f07dd67ef" containerName="main" Jul 17 22:07:33.541893 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.541870 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="59ce4675-257e-45ac-b0cd-aa2f07dd67ef" containerName="main" Jul 17 22:07:33.541893 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.541893 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" containerName="main" Jul 17 22:07:33.541993 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.541898 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" containerName="main" Jul 17 22:07:33.541993 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.541956 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="59ce4675-257e-45ac-b0cd-aa2f07dd67ef" containerName="main" Jul 17 22:07:33.541993 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.541965 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ac4bbd-e7ce-4b13-8fe9-84ee37c8a69d" containerName="main" Jul 17 22:07:33.545125 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.545104 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.548996 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.548972 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2a645a\"/\"openshift-service-ca.crt\"" Jul 17 22:07:33.549128 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.548998 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2a645a\"/\"kube-root-ca.crt\"" Jul 17 22:07:33.549128 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.548974 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2a645a\"/\"default-dockercfg-2l4p5\"" Jul 17 22:07:33.549128 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.548973 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2a645a\"/\"auth-test-model-2a645a-kserve-self-signed-certs\"" Jul 17 22:07:33.554589 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.554564 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25"] Jul 17 22:07:33.638207 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.638173 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:33.665499 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.665473 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cb16766-bed2-427e-9454-acd810fd3165-tls-certs\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.665647 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.665526 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgmfn\" (UniqueName: \"kubernetes.io/projected/6cb16766-bed2-427e-9454-acd810fd3165-kube-api-access-fgmfn\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.665647 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.665610 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-home\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.665647 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.665642 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-tmp-dir\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.665772 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.665663 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-dshm\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.665772 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.665705 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-model-cache\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.766175 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.766133 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-model-cache\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.766389 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.766185 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cb16766-bed2-427e-9454-acd810fd3165-tls-certs\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.766389 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.766240 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgmfn\" (UniqueName: \"kubernetes.io/projected/6cb16766-bed2-427e-9454-acd810fd3165-kube-api-access-fgmfn\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.766389 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.766289 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-home\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.766389 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.766326 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-tmp-dir\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.766389 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.766352 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-dshm\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.766677 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.766599 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-model-cache\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.766738 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.766692 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-tmp-dir\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.766738 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.766727 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-home\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.768757 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.768737 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-dshm\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.768878 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.768859 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cb16766-bed2-427e-9454-acd810fd3165-tls-certs\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.776752 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.776733 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgmfn\" (UniqueName: \"kubernetes.io/projected/6cb16766-bed2-427e-9454-acd810fd3165-kube-api-access-fgmfn\") pod \"auth-test-model-2a645a-kserve-f84f66c9-wsx25\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.855901 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.855815 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:33.998340 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:33.998311 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25"] Jul 17 22:07:34.000327 ip-10-0-131-188 kubenswrapper[2554]: W0717 22:07:34.000301 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cb16766_bed2_427e_9454_acd810fd3165.slice/crio-81a02c9b26d1c7d5899b8df57c03f94858ef51fbfa33c41dd74df37f72350f32 WatchSource:0}: Error finding container 81a02c9b26d1c7d5899b8df57c03f94858ef51fbfa33c41dd74df37f72350f32: Status 404 returned error can't find the container with id 81a02c9b26d1c7d5899b8df57c03f94858ef51fbfa33c41dd74df37f72350f32 Jul 17 22:07:34.054117 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:34.054078 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" event={"ID":"6cb16766-bed2-427e-9454-acd810fd3165","Type":"ContainerStarted","Data":"81a02c9b26d1c7d5899b8df57c03f94858ef51fbfa33c41dd74df37f72350f32"} Jul 17 22:07:34.667443 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:34.667413 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:34.767229 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:34.767192 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:35.060962 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:35.060921 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" event={"ID":"6cb16766-bed2-427e-9454-acd810fd3165","Type":"ContainerStarted","Data":"e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44"} Jul 17 22:07:35.079702 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:35.079640 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" podStartSLOduration=2.07962081 podStartE2EDuration="2.07962081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 22:07:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 22:07:35.079213587 +0000 UTC m=+3035.848561566" watchObservedRunningTime="2026-07-17 22:07:35.07962081 +0000 UTC m=+3035.848968792" Jul 17 22:07:38.639187 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:38.639147 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:07:42.962460 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:42.962420 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:43.063398 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:43.063365 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:07:43.856960 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:43.856856 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:43.856960 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:43.856924 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:43.869618 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:43.869587 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:07:44.123038 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:07:44.122954 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:08:05.050839 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.050807 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks"] Jul 17 22:08:05.054803 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.054783 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.057829 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.057802 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"kube-root-ca.crt\"" Jul 17 22:08:05.057950 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.057850 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"openshift-service-ca.crt\"" Jul 17 22:08:05.059166 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.059149 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"auth-test-model-b96f25-kserve-self-signed-certs\"" Jul 17 22:08:05.059166 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.059160 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"default-dockercfg-q79qw\"" Jul 17 22:08:05.069478 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.069456 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks"] Jul 17 22:08:05.084169 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.084144 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:08:05.158007 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.157977 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-dshm\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.158007 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.158009 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-model-cache\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.158255 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.158068 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tls-certs\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.158255 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.158091 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-home\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.158255 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.158116 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tmp-dir\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.158255 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.158217 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlqx9\" (UniqueName: \"kubernetes.io/projected/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-kube-api-access-rlqx9\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.258962 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.258929 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tls-certs\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.259158 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.258978 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-home\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.259158 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.259005 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tmp-dir\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.259158 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.259062 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlqx9\" (UniqueName: \"kubernetes.io/projected/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-kube-api-access-rlqx9\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.259158 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.259107 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-dshm\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.259382 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.259237 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-model-cache\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.259443 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.259421 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tmp-dir\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.259500 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.259453 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-home\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.259595 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.259577 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-model-cache\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.261597 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.261576 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-dshm\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.261710 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.261697 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tls-certs\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.268687 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.268664 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlqx9\" (UniqueName: \"kubernetes.io/projected/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-kube-api-access-rlqx9\") pod \"auth-test-model-b96f25-kserve-9c6b58c5f-mprks\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.365130 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.365026 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:05.506089 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:05.506062 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks"] Jul 17 22:08:05.508118 ip-10-0-131-188 kubenswrapper[2554]: W0717 22:08:05.508080 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e3dec30_b0da_42f8_a79f_c9d72bc44f1b.slice/crio-55a0b75913b9fc870fdfff918e7702cdbb3e3a9c4cbdbd9215b95ff3a1e9d325 WatchSource:0}: Error finding container 55a0b75913b9fc870fdfff918e7702cdbb3e3a9c4cbdbd9215b95ff3a1e9d325: Status 404 returned error can't find the container with id 55a0b75913b9fc870fdfff918e7702cdbb3e3a9c4cbdbd9215b95ff3a1e9d325 Jul 17 22:08:06.216789 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:06.216754 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" event={"ID":"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b","Type":"ContainerStarted","Data":"d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8"} Jul 17 22:08:06.216789 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:06.216794 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" event={"ID":"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b","Type":"ContainerStarted","Data":"55a0b75913b9fc870fdfff918e7702cdbb3e3a9c4cbdbd9215b95ff3a1e9d325"} Jul 17 22:08:06.233448 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:06.233399 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" podStartSLOduration=1.233383029 podStartE2EDuration="1.233383029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 22:08:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 22:08:06.232995977 +0000 UTC m=+3067.002343958" watchObservedRunningTime="2026-07-17 22:08:06.233383029 +0000 UTC m=+3067.002731011" Jul 17 22:08:08.840869 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:08.840834 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:08:15.365804 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:15.365766 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:15.366290 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:15.365822 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:15.378418 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:15.378390 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:16.279917 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:16.279881 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:08:32.340744 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:32.340705 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:08:52.442137 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:08:52.442097 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:09:07.335961 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:09:07.335925 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:09:22.535143 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:09:22.535059 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:09:37.241901 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:09:37.241859 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:09:51.743188 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:09:51.743147 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:10:05.425268 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:05.425234 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25"] Jul 17 22:10:05.472995 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:05.472953 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:10:06.347012 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:06.346964 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks"] Jul 17 22:10:06.369573 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:06.369532 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:10:10.046390 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:10.046358 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:10:20.447653 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.447588 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" podUID="6cb16766-bed2-427e-9454-acd810fd3165" containerName="main" containerID="cri-o://e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44" gracePeriod=45 Jul 17 22:10:20.714927 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.714902 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:10:20.837307 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.837271 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-home\") pod \"6cb16766-bed2-427e-9454-acd810fd3165\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " Jul 17 22:10:20.837470 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.837338 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgmfn\" (UniqueName: \"kubernetes.io/projected/6cb16766-bed2-427e-9454-acd810fd3165-kube-api-access-fgmfn\") pod \"6cb16766-bed2-427e-9454-acd810fd3165\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " Jul 17 22:10:20.837470 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.837377 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-dshm\") pod \"6cb16766-bed2-427e-9454-acd810fd3165\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " Jul 17 22:10:20.837470 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.837421 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cb16766-bed2-427e-9454-acd810fd3165-tls-certs\") pod \"6cb16766-bed2-427e-9454-acd810fd3165\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " Jul 17 22:10:20.837470 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.837454 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-tmp-dir\") pod \"6cb16766-bed2-427e-9454-acd810fd3165\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " Jul 17 22:10:20.837470 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.837470 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-model-cache\") pod \"6cb16766-bed2-427e-9454-acd810fd3165\" (UID: \"6cb16766-bed2-427e-9454-acd810fd3165\") " Jul 17 22:10:20.837751 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.837611 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-home" (OuterVolumeSpecName: "home") pod "6cb16766-bed2-427e-9454-acd810fd3165" (UID: "6cb16766-bed2-427e-9454-acd810fd3165"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:10:20.837815 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.837790 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6cb16766-bed2-427e-9454-acd810fd3165" (UID: "6cb16766-bed2-427e-9454-acd810fd3165"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:10:20.837869 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.837820 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-model-cache" (OuterVolumeSpecName: "model-cache") pod "6cb16766-bed2-427e-9454-acd810fd3165" (UID: "6cb16766-bed2-427e-9454-acd810fd3165"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:10:20.837869 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.837856 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-home\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:20.839778 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.839757 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-dshm" (OuterVolumeSpecName: "dshm") pod "6cb16766-bed2-427e-9454-acd810fd3165" (UID: "6cb16766-bed2-427e-9454-acd810fd3165"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:10:20.839778 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.839770 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cb16766-bed2-427e-9454-acd810fd3165-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6cb16766-bed2-427e-9454-acd810fd3165" (UID: "6cb16766-bed2-427e-9454-acd810fd3165"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 22:10:20.839973 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.839823 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cb16766-bed2-427e-9454-acd810fd3165-kube-api-access-fgmfn" (OuterVolumeSpecName: "kube-api-access-fgmfn") pod "6cb16766-bed2-427e-9454-acd810fd3165" (UID: "6cb16766-bed2-427e-9454-acd810fd3165"). InnerVolumeSpecName "kube-api-access-fgmfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 22:10:20.918045 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.918002 2554 generic.go:356] "Generic (PLEG): container finished" podID="6cb16766-bed2-427e-9454-acd810fd3165" containerID="e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44" exitCode=0 Jul 17 22:10:20.918045 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.918045 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" Jul 17 22:10:20.918302 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.918082 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" event={"ID":"6cb16766-bed2-427e-9454-acd810fd3165","Type":"ContainerDied","Data":"e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44"} Jul 17 22:10:20.918302 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.918147 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25" event={"ID":"6cb16766-bed2-427e-9454-acd810fd3165","Type":"ContainerDied","Data":"81a02c9b26d1c7d5899b8df57c03f94858ef51fbfa33c41dd74df37f72350f32"} Jul 17 22:10:20.918302 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.918172 2554 scope.go:117] "RemoveContainer" containerID="e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44" Jul 17 22:10:20.928290 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.928270 2554 scope.go:117] "RemoveContainer" containerID="e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44" Jul 17 22:10:20.928596 ip-10-0-131-188 kubenswrapper[2554]: E0717 22:10:20.928575 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44\": container with ID starting with e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44 not found: ID does not exist" containerID="e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44" Jul 17 22:10:20.928710 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.928608 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44"} err="failed to get container status \"e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44\": rpc error: code = NotFound desc = could not find container \"e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44\": container with ID starting with e0662f95af2328fd9134e7074e8083c72fe64de699bfcad7616523fe173c5c44 not found: ID does not exist" Jul 17 22:10:20.938569 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.938519 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgmfn\" (UniqueName: \"kubernetes.io/projected/6cb16766-bed2-427e-9454-acd810fd3165-kube-api-access-fgmfn\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:20.938691 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.938573 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-dshm\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:20.938691 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.938586 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cb16766-bed2-427e-9454-acd810fd3165-tls-certs\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:20.938691 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.938595 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-tmp-dir\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:20.938691 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.938603 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cb16766-bed2-427e-9454-acd810fd3165-model-cache\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:20.940085 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.940064 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25"] Jul 17 22:10:20.943956 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:20.943934 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/auth-test-model-2a645a-kserve-f84f66c9-wsx25"] Jul 17 22:10:21.367995 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.367956 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" podUID="2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" containerName="main" containerID="cri-o://d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8" gracePeriod=45 Jul 17 22:10:21.631447 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.631382 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:10:21.720262 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.720226 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cb16766-bed2-427e-9454-acd810fd3165" path="/var/lib/kubelet/pods/6cb16766-bed2-427e-9454-acd810fd3165/volumes" Jul 17 22:10:21.748513 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.748475 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-dshm\") pod \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " Jul 17 22:10:21.748513 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.748514 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tls-certs\") pod \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " Jul 17 22:10:21.748822 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.748570 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tmp-dir\") pod \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " Jul 17 22:10:21.748822 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.748601 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlqx9\" (UniqueName: \"kubernetes.io/projected/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-kube-api-access-rlqx9\") pod \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " Jul 17 22:10:21.748822 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.748631 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-model-cache\") pod \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " Jul 17 22:10:21.748822 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.748648 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-home\") pod \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\" (UID: \"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b\") " Jul 17 22:10:21.749041 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.748876 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" (UID: "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:10:21.749041 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.748939 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-model-cache" (OuterVolumeSpecName: "model-cache") pod "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" (UID: "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:10:21.749143 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.749100 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-home" (OuterVolumeSpecName: "home") pod "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" (UID: "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:10:21.749200 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.749168 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tmp-dir\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:21.749200 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.749191 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-model-cache\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:21.751005 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.750983 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-kube-api-access-rlqx9" (OuterVolumeSpecName: "kube-api-access-rlqx9") pod "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" (UID: "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b"). InnerVolumeSpecName "kube-api-access-rlqx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 22:10:21.751364 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.751349 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-dshm" (OuterVolumeSpecName: "dshm") pod "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" (UID: "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:10:21.751457 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.751439 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" (UID: "2e3dec30-b0da-42f8-a79f-c9d72bc44f1b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 22:10:21.849684 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.849652 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-dshm\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:21.849684 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.849678 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-tls-certs\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:21.849684 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.849689 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlqx9\" (UniqueName: \"kubernetes.io/projected/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-kube-api-access-rlqx9\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:21.849928 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.849697 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b-home\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:10:21.929615 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.929499 2554 generic.go:356] "Generic (PLEG): container finished" podID="2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" containerID="d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8" exitCode=0 Jul 17 22:10:21.929615 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.929587 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" Jul 17 22:10:21.929615 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.929592 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" event={"ID":"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b","Type":"ContainerDied","Data":"d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8"} Jul 17 22:10:21.929929 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.929638 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks" event={"ID":"2e3dec30-b0da-42f8-a79f-c9d72bc44f1b","Type":"ContainerDied","Data":"55a0b75913b9fc870fdfff918e7702cdbb3e3a9c4cbdbd9215b95ff3a1e9d325"} Jul 17 22:10:21.929929 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.929661 2554 scope.go:117] "RemoveContainer" containerID="d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8" Jul 17 22:10:21.940157 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.940138 2554 scope.go:117] "RemoveContainer" containerID="d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8" Jul 17 22:10:21.940441 ip-10-0-131-188 kubenswrapper[2554]: E0717 22:10:21.940418 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8\": container with ID starting with d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8 not found: ID does not exist" containerID="d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8" Jul 17 22:10:21.940512 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.940452 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8"} err="failed to get container status \"d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8\": rpc error: code = NotFound desc = could not find container \"d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8\": container with ID starting with d8b8b32c601dbc3128faf8cd47d6563f701af42c60061d1fd627a9959fa114d8 not found: ID does not exist" Jul 17 22:10:21.952053 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.952025 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks"] Jul 17 22:10:21.956067 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:21.956043 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/auth-test-model-b96f25-kserve-9c6b58c5f-mprks"] Jul 17 22:10:23.720240 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:23.720203 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" path="/var/lib/kubelet/pods/2e3dec30-b0da-42f8-a79f-c9d72bc44f1b/volumes" Jul 17 22:10:26.940289 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.940252 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt"] Jul 17 22:10:26.940902 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.940872 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cb16766-bed2-427e-9454-acd810fd3165" containerName="main" Jul 17 22:10:26.940902 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.940895 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cb16766-bed2-427e-9454-acd810fd3165" containerName="main" Jul 17 22:10:26.941042 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.940922 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" containerName="main" Jul 17 22:10:26.941042 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.940931 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" containerName="main" Jul 17 22:10:26.941042 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.941024 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cb16766-bed2-427e-9454-acd810fd3165" containerName="main" Jul 17 22:10:26.941189 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.941044 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e3dec30-b0da-42f8-a79f-c9d72bc44f1b" containerName="main" Jul 17 22:10:26.945855 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.945832 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:26.948556 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.948506 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2a645a\"/\"openshift-service-ca.crt\"" Jul 17 22:10:26.949711 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.949688 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2a645a\"/\"sub-test-model-2a645a-kserve-self-signed-certs\"" Jul 17 22:10:26.949840 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.949731 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2a645a\"/\"default-dockercfg-2l4p5\"" Jul 17 22:10:26.949840 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.949760 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2a645a\"/\"kube-root-ca.crt\"" Jul 17 22:10:26.953836 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.953812 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt"] Jul 17 22:10:26.961608 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:26.961567 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:10:27.097852 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.097815 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-home\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.098056 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.097870 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-dshm\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.098056 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.097953 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-tmp-dir\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.098056 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.097994 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49tr6\" (UniqueName: \"kubernetes.io/projected/b7abc321-3725-4589-ba08-174ed319a472-kube-api-access-49tr6\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.098056 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.098016 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-model-cache\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.098271 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.098103 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7abc321-3725-4589-ba08-174ed319a472-tls-certs\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.199672 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.199577 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-home\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.199672 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.199628 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-dshm\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.199672 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.199650 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-tmp-dir\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.199672 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.199673 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49tr6\" (UniqueName: \"kubernetes.io/projected/b7abc321-3725-4589-ba08-174ed319a472-kube-api-access-49tr6\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.199984 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.199690 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-model-cache\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.199984 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.199743 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7abc321-3725-4589-ba08-174ed319a472-tls-certs\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.200156 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.200124 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-model-cache\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.200238 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.200174 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-home\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.200238 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.200177 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-tmp-dir\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.201990 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.201972 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-dshm\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.202427 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.202405 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7abc321-3725-4589-ba08-174ed319a472-tls-certs\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.208029 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.208009 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49tr6\" (UniqueName: \"kubernetes.io/projected/b7abc321-3725-4589-ba08-174ed319a472-kube-api-access-49tr6\") pod \"sub-test-model-2a645a-kserve-55567cbd77-qqswt\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.257104 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.257063 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:27.400871 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.400744 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt"] Jul 17 22:10:27.404085 ip-10-0-131-188 kubenswrapper[2554]: W0717 22:10:27.404060 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7abc321_3725_4589_ba08_174ed319a472.slice/crio-45fe4cbe296546b541f6c00fbb23c6c156a28112d289dac484a4e7efcbfe11c8 WatchSource:0}: Error finding container 45fe4cbe296546b541f6c00fbb23c6c156a28112d289dac484a4e7efcbfe11c8: Status 404 returned error can't find the container with id 45fe4cbe296546b541f6c00fbb23c6c156a28112d289dac484a4e7efcbfe11c8 Jul 17 22:10:27.965185 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.965149 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" event={"ID":"b7abc321-3725-4589-ba08-174ed319a472","Type":"ContainerStarted","Data":"23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72"} Jul 17 22:10:27.965185 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.965196 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" event={"ID":"b7abc321-3725-4589-ba08-174ed319a472","Type":"ContainerStarted","Data":"45fe4cbe296546b541f6c00fbb23c6c156a28112d289dac484a4e7efcbfe11c8"} Jul 17 22:10:27.984121 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:27.984074 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" podStartSLOduration=1.9840574549999999 podStartE2EDuration="1.984057455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 22:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 22:10:27.982219365 +0000 UTC m=+3208.751567364" watchObservedRunningTime="2026-07-17 22:10:27.984057455 +0000 UTC m=+3208.753405432" Jul 17 22:10:30.838415 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:30.838372 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:10:35.876498 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:35.876462 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:10:35.976946 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:35.976912 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:10:37.258198 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:37.258162 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:37.258752 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:37.258442 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:37.270999 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:37.270975 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:10:38.027153 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:10:38.027125 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:11:04.040561 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.040510 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh"] Jul 17 22:11:04.045377 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.045356 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.049377 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.049347 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"sub-test-model-b96f25-kserve-self-signed-certs\"" Jul 17 22:11:04.049377 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.049367 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"default-dockercfg-q79qw\"" Jul 17 22:11:04.049615 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.049355 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"kube-root-ca.crt\"" Jul 17 22:11:04.049615 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.049367 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"openshift-service-ca.crt\"" Jul 17 22:11:04.052804 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.052781 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh"] Jul 17 22:11:04.080061 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.080028 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:11:04.132792 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.132757 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv6n9\" (UniqueName: \"kubernetes.io/projected/65c97ffa-4c5a-45f3-90c9-28c96f425551-kube-api-access-cv6n9\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.132792 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.132794 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-dshm\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.133008 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.132887 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65c97ffa-4c5a-45f3-90c9-28c96f425551-tls-certs\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.133008 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.132931 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-home\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.133090 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.133005 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-tmp-dir\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.133090 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.133059 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-model-cache\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.234434 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.234387 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65c97ffa-4c5a-45f3-90c9-28c96f425551-tls-certs\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.234660 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.234453 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-home\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.234660 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.234500 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-tmp-dir\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.234660 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.234535 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-model-cache\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.234660 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.234599 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv6n9\" (UniqueName: \"kubernetes.io/projected/65c97ffa-4c5a-45f3-90c9-28c96f425551-kube-api-access-cv6n9\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.234660 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.234623 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-dshm\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.234987 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.234961 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-tmp-dir\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.235054 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.235001 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-model-cache\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.235054 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.235014 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-home\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.237039 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.237011 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-dshm\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.237285 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.237269 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65c97ffa-4c5a-45f3-90c9-28c96f425551-tls-certs\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.243868 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.243842 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv6n9\" (UniqueName: \"kubernetes.io/projected/65c97ffa-4c5a-45f3-90c9-28c96f425551-kube-api-access-cv6n9\") pod \"sub-test-model-b96f25-kserve-7db55547c5-lw4jh\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.357123 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.357010 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:04.494724 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:04.494696 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh"] Jul 17 22:11:04.497212 ip-10-0-131-188 kubenswrapper[2554]: W0717 22:11:04.497182 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65c97ffa_4c5a_45f3_90c9_28c96f425551.slice/crio-12f22500775f354b9a30cbc59c4220d52bb07bea217e435834c9d87dc1c78136 WatchSource:0}: Error finding container 12f22500775f354b9a30cbc59c4220d52bb07bea217e435834c9d87dc1c78136: Status 404 returned error can't find the container with id 12f22500775f354b9a30cbc59c4220d52bb07bea217e435834c9d87dc1c78136 Jul 17 22:11:05.158026 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:05.157991 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" event={"ID":"65c97ffa-4c5a-45f3-90c9-28c96f425551","Type":"ContainerStarted","Data":"0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca"} Jul 17 22:11:05.158424 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:05.158035 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" event={"ID":"65c97ffa-4c5a-45f3-90c9-28c96f425551","Type":"ContainerStarted","Data":"12f22500775f354b9a30cbc59c4220d52bb07bea217e435834c9d87dc1c78136"} Jul 17 22:11:05.175918 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:05.175868 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" podStartSLOduration=1.175853798 podStartE2EDuration="1.175853798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 22:11:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 22:11:05.17478496 +0000 UTC m=+3245.944132938" watchObservedRunningTime="2026-07-17 22:11:05.175853798 +0000 UTC m=+3245.945201775" Jul 17 22:11:08.850030 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:08.849991 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:11:14.357995 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:14.357944 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:14.357995 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:14.358015 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:14.370861 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:14.370832 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:15.219590 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:15.219560 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:23.743905 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:23.743868 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:11:41.978820 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:41.978783 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt"] Jul 17 22:11:41.999981 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:41.999953 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:11:42.915440 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:42.915404 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh"] Jul 17 22:11:42.930492 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:42.930464 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:11:44.842840 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:44.842805 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:11:51.301207 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.301174 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg"] Jul 17 22:11:51.304921 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.304899 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.307284 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.307256 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2a645a\"/\"rate-test-model-2a645a-kserve-self-signed-certs\"" Jul 17 22:11:51.313023 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.312988 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg"] Jul 17 22:11:51.396038 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.396006 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-dshm\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.396234 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.396048 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-model-cache\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.396234 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.396068 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwvdz\" (UniqueName: \"kubernetes.io/projected/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-kube-api-access-nwvdz\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.396368 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.396230 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-home\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.396368 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.396271 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tmp-dir\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.396368 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.396318 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tls-certs\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.497445 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.497406 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-home\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.497445 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.497443 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tmp-dir\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.497763 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.497466 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tls-certs\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.497763 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.497517 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-dshm\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.497763 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.497584 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-model-cache\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.497763 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.497610 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwvdz\" (UniqueName: \"kubernetes.io/projected/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-kube-api-access-nwvdz\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.497975 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.497934 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-home\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.497975 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.497949 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tmp-dir\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.498054 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.498032 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-model-cache\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.500051 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.500033 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-dshm\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.500187 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.500172 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tls-certs\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.506069 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.506050 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwvdz\" (UniqueName: \"kubernetes.io/projected/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-kube-api-access-nwvdz\") pod \"rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.617459 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.617363 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:11:51.766850 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:51.766823 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg"] Jul 17 22:11:51.769535 ip-10-0-131-188 kubenswrapper[2554]: W0717 22:11:51.769504 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf70cd1d5_cb51_496c_a2d2_face4d7af1a5.slice/crio-7f2d5f8abfde56f6b1e8caa246d0772f4f3a7642238e337b0c1101ecc781b30a WatchSource:0}: Error finding container 7f2d5f8abfde56f6b1e8caa246d0772f4f3a7642238e337b0c1101ecc781b30a: Status 404 returned error can't find the container with id 7f2d5f8abfde56f6b1e8caa246d0772f4f3a7642238e337b0c1101ecc781b30a Jul 17 22:11:52.113672 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:52.113635 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:11:52.396884 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:52.396794 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" event={"ID":"f70cd1d5-cb51-496c-a2d2-face4d7af1a5","Type":"ContainerStarted","Data":"b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147"} Jul 17 22:11:52.396884 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:52.396836 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" event={"ID":"f70cd1d5-cb51-496c-a2d2-face4d7af1a5","Type":"ContainerStarted","Data":"7f2d5f8abfde56f6b1e8caa246d0772f4f3a7642238e337b0c1101ecc781b30a"} Jul 17 22:11:52.414904 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:52.414849 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" podStartSLOduration=1.414830278 podStartE2EDuration="1.414830278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 22:11:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 22:11:52.414439356 +0000 UTC m=+3293.183787335" watchObservedRunningTime="2026-07-17 22:11:52.414830278 +0000 UTC m=+3293.184178255" Jul 17 22:11:53.997452 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:53.997407 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:11:54.090155 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:54.090118 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:11:57.000486 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.000447 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" podUID="b7abc321-3725-4589-ba08-174ed319a472" containerName="main" containerID="cri-o://23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72" gracePeriod=45 Jul 17 22:11:57.264414 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.264349 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:11:57.355990 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.355954 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7abc321-3725-4589-ba08-174ed319a472-tls-certs\") pod \"b7abc321-3725-4589-ba08-174ed319a472\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " Jul 17 22:11:57.356194 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.356046 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-home\") pod \"b7abc321-3725-4589-ba08-174ed319a472\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " Jul 17 22:11:57.356194 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.356062 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-dshm\") pod \"b7abc321-3725-4589-ba08-174ed319a472\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " Jul 17 22:11:57.356194 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.356096 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49tr6\" (UniqueName: \"kubernetes.io/projected/b7abc321-3725-4589-ba08-174ed319a472-kube-api-access-49tr6\") pod \"b7abc321-3725-4589-ba08-174ed319a472\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " Jul 17 22:11:57.356194 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.356136 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-tmp-dir\") pod \"b7abc321-3725-4589-ba08-174ed319a472\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " Jul 17 22:11:57.356194 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.356164 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-model-cache\") pod \"b7abc321-3725-4589-ba08-174ed319a472\" (UID: \"b7abc321-3725-4589-ba08-174ed319a472\") " Jul 17 22:11:57.356456 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.356354 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-home" (OuterVolumeSpecName: "home") pod "b7abc321-3725-4589-ba08-174ed319a472" (UID: "b7abc321-3725-4589-ba08-174ed319a472"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:11:57.356456 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.356370 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b7abc321-3725-4589-ba08-174ed319a472" (UID: "b7abc321-3725-4589-ba08-174ed319a472"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:11:57.356537 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.356503 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-model-cache" (OuterVolumeSpecName: "model-cache") pod "b7abc321-3725-4589-ba08-174ed319a472" (UID: "b7abc321-3725-4589-ba08-174ed319a472"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:11:57.356537 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.356508 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-tmp-dir\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:57.356648 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.356567 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-home\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:57.358265 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.358244 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-dshm" (OuterVolumeSpecName: "dshm") pod "b7abc321-3725-4589-ba08-174ed319a472" (UID: "b7abc321-3725-4589-ba08-174ed319a472"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:11:57.358360 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.358346 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7abc321-3725-4589-ba08-174ed319a472-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b7abc321-3725-4589-ba08-174ed319a472" (UID: "b7abc321-3725-4589-ba08-174ed319a472"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 22:11:57.358473 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.358459 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7abc321-3725-4589-ba08-174ed319a472-kube-api-access-49tr6" (OuterVolumeSpecName: "kube-api-access-49tr6") pod "b7abc321-3725-4589-ba08-174ed319a472" (UID: "b7abc321-3725-4589-ba08-174ed319a472"). InnerVolumeSpecName "kube-api-access-49tr6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 22:11:57.422733 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.422701 2554 generic.go:356] "Generic (PLEG): container finished" podID="b7abc321-3725-4589-ba08-174ed319a472" containerID="23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72" exitCode=0 Jul 17 22:11:57.422900 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.422741 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" Jul 17 22:11:57.422900 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.422765 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" event={"ID":"b7abc321-3725-4589-ba08-174ed319a472","Type":"ContainerDied","Data":"23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72"} Jul 17 22:11:57.422900 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.422796 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt" event={"ID":"b7abc321-3725-4589-ba08-174ed319a472","Type":"ContainerDied","Data":"45fe4cbe296546b541f6c00fbb23c6c156a28112d289dac484a4e7efcbfe11c8"} Jul 17 22:11:57.422900 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.422812 2554 scope.go:117] "RemoveContainer" containerID="23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72" Jul 17 22:11:57.432639 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.432619 2554 scope.go:117] "RemoveContainer" containerID="23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72" Jul 17 22:11:57.432869 ip-10-0-131-188 kubenswrapper[2554]: E0717 22:11:57.432849 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72\": container with ID starting with 23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72 not found: ID does not exist" containerID="23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72" Jul 17 22:11:57.432932 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.432878 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72"} err="failed to get container status \"23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72\": rpc error: code = NotFound desc = could not find container \"23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72\": container with ID starting with 23606f4b2f444ca478fd7dfec7771043243a05d451927c022873b51111c3da72 not found: ID does not exist" Jul 17 22:11:57.444093 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.444067 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt"] Jul 17 22:11:57.448046 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.448024 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/sub-test-model-2a645a-kserve-55567cbd77-qqswt"] Jul 17 22:11:57.457414 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.457393 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49tr6\" (UniqueName: \"kubernetes.io/projected/b7abc321-3725-4589-ba08-174ed319a472-kube-api-access-49tr6\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:57.457414 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.457413 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-model-cache\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:57.457540 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.457423 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7abc321-3725-4589-ba08-174ed319a472-tls-certs\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:57.457540 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.457431 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7abc321-3725-4589-ba08-174ed319a472-dshm\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:57.720474 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.720445 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7abc321-3725-4589-ba08-174ed319a472" path="/var/lib/kubelet/pods/b7abc321-3725-4589-ba08-174ed319a472/volumes" Jul 17 22:11:57.936716 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:57.936676 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" podUID="65c97ffa-4c5a-45f3-90c9-28c96f425551" containerName="main" containerID="cri-o://0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca" gracePeriod=45 Jul 17 22:11:58.196903 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.196883 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:58.265029 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.264933 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65c97ffa-4c5a-45f3-90c9-28c96f425551-tls-certs\") pod \"65c97ffa-4c5a-45f3-90c9-28c96f425551\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " Jul 17 22:11:58.265029 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.265007 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv6n9\" (UniqueName: \"kubernetes.io/projected/65c97ffa-4c5a-45f3-90c9-28c96f425551-kube-api-access-cv6n9\") pod \"65c97ffa-4c5a-45f3-90c9-28c96f425551\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " Jul 17 22:11:58.265029 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.265033 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-tmp-dir\") pod \"65c97ffa-4c5a-45f3-90c9-28c96f425551\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " Jul 17 22:11:58.265379 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.265071 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-home\") pod \"65c97ffa-4c5a-45f3-90c9-28c96f425551\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " Jul 17 22:11:58.265379 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.265102 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-model-cache\") pod \"65c97ffa-4c5a-45f3-90c9-28c96f425551\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " Jul 17 22:11:58.265379 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.265131 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-dshm\") pod \"65c97ffa-4c5a-45f3-90c9-28c96f425551\" (UID: \"65c97ffa-4c5a-45f3-90c9-28c96f425551\") " Jul 17 22:11:58.265379 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.265340 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "65c97ffa-4c5a-45f3-90c9-28c96f425551" (UID: "65c97ffa-4c5a-45f3-90c9-28c96f425551"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:11:58.265558 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.265378 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-home" (OuterVolumeSpecName: "home") pod "65c97ffa-4c5a-45f3-90c9-28c96f425551" (UID: "65c97ffa-4c5a-45f3-90c9-28c96f425551"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:11:58.265602 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.265539 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-model-cache" (OuterVolumeSpecName: "model-cache") pod "65c97ffa-4c5a-45f3-90c9-28c96f425551" (UID: "65c97ffa-4c5a-45f3-90c9-28c96f425551"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:11:58.267341 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.267322 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-dshm" (OuterVolumeSpecName: "dshm") pod "65c97ffa-4c5a-45f3-90c9-28c96f425551" (UID: "65c97ffa-4c5a-45f3-90c9-28c96f425551"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:11:58.267448 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.267435 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c97ffa-4c5a-45f3-90c9-28c96f425551-kube-api-access-cv6n9" (OuterVolumeSpecName: "kube-api-access-cv6n9") pod "65c97ffa-4c5a-45f3-90c9-28c96f425551" (UID: "65c97ffa-4c5a-45f3-90c9-28c96f425551"). InnerVolumeSpecName "kube-api-access-cv6n9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 22:11:58.267693 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.267669 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c97ffa-4c5a-45f3-90c9-28c96f425551-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "65c97ffa-4c5a-45f3-90c9-28c96f425551" (UID: "65c97ffa-4c5a-45f3-90c9-28c96f425551"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 22:11:58.366657 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.366622 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv6n9\" (UniqueName: \"kubernetes.io/projected/65c97ffa-4c5a-45f3-90c9-28c96f425551-kube-api-access-cv6n9\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:58.366657 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.366652 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-tmp-dir\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:58.366657 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.366664 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-home\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:58.366901 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.366673 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-model-cache\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:58.366901 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.366680 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65c97ffa-4c5a-45f3-90c9-28c96f425551-dshm\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:58.366901 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.366688 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65c97ffa-4c5a-45f3-90c9-28c96f425551-tls-certs\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:11:58.435990 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.435950 2554 generic.go:356] "Generic (PLEG): container finished" podID="65c97ffa-4c5a-45f3-90c9-28c96f425551" containerID="0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca" exitCode=0 Jul 17 22:11:58.436183 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.436005 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" Jul 17 22:11:58.436183 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.436029 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" event={"ID":"65c97ffa-4c5a-45f3-90c9-28c96f425551","Type":"ContainerDied","Data":"0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca"} Jul 17 22:11:58.436183 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.436078 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh" event={"ID":"65c97ffa-4c5a-45f3-90c9-28c96f425551","Type":"ContainerDied","Data":"12f22500775f354b9a30cbc59c4220d52bb07bea217e435834c9d87dc1c78136"} Jul 17 22:11:58.436183 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.436100 2554 scope.go:117] "RemoveContainer" containerID="0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca" Jul 17 22:11:58.446068 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.446049 2554 scope.go:117] "RemoveContainer" containerID="0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca" Jul 17 22:11:58.446337 ip-10-0-131-188 kubenswrapper[2554]: E0717 22:11:58.446318 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca\": container with ID starting with 0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca not found: ID does not exist" containerID="0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca" Jul 17 22:11:58.446382 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.446346 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca"} err="failed to get container status \"0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca\": rpc error: code = NotFound desc = could not find container \"0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca\": container with ID starting with 0cd077543aa285ec314b83bcbf29ab093f89b81d8306444b6be5daa56d485fca not found: ID does not exist" Jul 17 22:11:58.457909 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.457876 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh"] Jul 17 22:11:58.460580 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:58.460559 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/sub-test-model-b96f25-kserve-7db55547c5-lw4jh"] Jul 17 22:11:59.720481 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:59.720444 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65c97ffa-4c5a-45f3-90c9-28c96f425551" path="/var/lib/kubelet/pods/65c97ffa-4c5a-45f3-90c9-28c96f425551/volumes" Jul 17 22:11:59.878070 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:59.878039 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 22:11:59.885996 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:11:59.885971 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 22:12:01.617469 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:01.617426 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:12:01.617931 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:01.617480 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:12:01.634201 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:01.634165 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:12:02.470593 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:02.470564 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:12:07.431445 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:07.431412 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:12:14.442472 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.442436 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n"] Jul 17 22:12:14.442900 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.442886 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7abc321-3725-4589-ba08-174ed319a472" containerName="main" Jul 17 22:12:14.442943 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.442902 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7abc321-3725-4589-ba08-174ed319a472" containerName="main" Jul 17 22:12:14.442943 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.442917 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c97ffa-4c5a-45f3-90c9-28c96f425551" containerName="main" Jul 17 22:12:14.442943 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.442926 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c97ffa-4c5a-45f3-90c9-28c96f425551" containerName="main" Jul 17 22:12:14.443040 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.442993 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7abc321-3725-4589-ba08-174ed319a472" containerName="main" Jul 17 22:12:14.443040 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.443010 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="65c97ffa-4c5a-45f3-90c9-28c96f425551" containerName="main" Jul 17 22:12:14.446124 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.446102 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.448830 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.448810 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"kube-root-ca.crt\"" Jul 17 22:12:14.449969 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.449945 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"default-dockercfg-q79qw\"" Jul 17 22:12:14.450103 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.449974 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"openshift-service-ca.crt\"" Jul 17 22:12:14.450103 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.449980 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b96f25\"/\"rate-test-model-b96f25-kserve-self-signed-certs\"" Jul 17 22:12:14.456751 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.456731 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n"] Jul 17 22:12:14.473445 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.473414 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:12:14.526333 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.526302 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k5qd\" (UniqueName: \"kubernetes.io/projected/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-kube-api-access-7k5qd\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.526512 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.526358 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-model-cache\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.526512 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.526420 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tmp-dir\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.526512 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.526448 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tls-certs\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.526700 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.526535 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-dshm\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.526700 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.526586 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-home\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.628049 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.628013 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k5qd\" (UniqueName: \"kubernetes.io/projected/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-kube-api-access-7k5qd\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.628258 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.628066 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-model-cache\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.628258 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.628191 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tmp-dir\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.628258 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.628224 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tls-certs\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.628420 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.628297 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-dshm\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.628420 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.628326 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-home\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.628508 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.628460 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-model-cache\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.628604 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.628579 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tmp-dir\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.628751 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.628726 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-home\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.630752 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.630732 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-dshm\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.630990 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.630973 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tls-certs\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.637271 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.637239 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k5qd\" (UniqueName: \"kubernetes.io/projected/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-kube-api-access-7k5qd\") pod \"rate-test-model-b96f25-kserve-77c977f679-zhg9n\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.757058 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.756960 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:14.898642 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:14.898601 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n"] Jul 17 22:12:14.900476 ip-10-0-131-188 kubenswrapper[2554]: W0717 22:12:14.900448 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bea0f52_7a00_4a27_b1a9_b8cc9218ccd4.slice/crio-7ffae6a9d27decf0708e09e20c4da69319029968ba5e27bab8876f7ab74f7f02 WatchSource:0}: Error finding container 7ffae6a9d27decf0708e09e20c4da69319029968ba5e27bab8876f7ab74f7f02: Status 404 returned error can't find the container with id 7ffae6a9d27decf0708e09e20c4da69319029968ba5e27bab8876f7ab74f7f02 Jul 17 22:12:15.526067 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:15.526033 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" event={"ID":"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4","Type":"ContainerStarted","Data":"3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f"} Jul 17 22:12:15.526456 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:15.526083 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" event={"ID":"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4","Type":"ContainerStarted","Data":"7ffae6a9d27decf0708e09e20c4da69319029968ba5e27bab8876f7ab74f7f02"} Jul 17 22:12:15.544974 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:15.544925 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" podStartSLOduration=1.5449096249999998 podStartE2EDuration="1.544909625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 22:12:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 22:12:15.543162159 +0000 UTC m=+3316.312510138" watchObservedRunningTime="2026-07-17 22:12:15.544909625 +0000 UTC m=+3316.314257629" Jul 17 22:12:19.442512 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:19.442477 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:12:24.757477 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:24.757370 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:24.757477 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:24.757430 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:24.769959 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:24.769923 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:25.587112 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:25.587082 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:12:35.436316 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:35.436276 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:12:36.646677 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:36.646641 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:12:41.043511 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:41.043473 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:12:43.858338 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:43.858240 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:12:43.957207 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:12:43.957156 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:04.632086 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:04.632042 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:13:09.356770 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:09.356734 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg"] Jul 17 22:13:09.382120 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:09.382082 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:10.223518 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:10.223484 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n"] Jul 17 22:13:10.250717 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:10.250673 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:21.240403 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:21.240364 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:13:24.377911 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.377863 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" podUID="f70cd1d5-cb51-496c-a2d2-face4d7af1a5" containerName="main" containerID="cri-o://b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147" gracePeriod=45 Jul 17 22:13:24.642970 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.642902 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:13:24.783676 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.783643 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-home\") pod \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " Jul 17 22:13:24.783893 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.783739 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-dshm\") pod \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " Jul 17 22:13:24.783893 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.783765 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwvdz\" (UniqueName: \"kubernetes.io/projected/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-kube-api-access-nwvdz\") pod \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " Jul 17 22:13:24.783893 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.783787 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-model-cache\") pod \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " Jul 17 22:13:24.783893 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.783811 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tls-certs\") pod \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " Jul 17 22:13:24.783893 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.783881 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tmp-dir\") pod \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\" (UID: \"f70cd1d5-cb51-496c-a2d2-face4d7af1a5\") " Jul 17 22:13:24.784144 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.783906 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-home" (OuterVolumeSpecName: "home") pod "f70cd1d5-cb51-496c-a2d2-face4d7af1a5" (UID: "f70cd1d5-cb51-496c-a2d2-face4d7af1a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:13:24.784144 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.784051 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-model-cache" (OuterVolumeSpecName: "model-cache") pod "f70cd1d5-cb51-496c-a2d2-face4d7af1a5" (UID: "f70cd1d5-cb51-496c-a2d2-face4d7af1a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:13:24.784262 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.784191 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f70cd1d5-cb51-496c-a2d2-face4d7af1a5" (UID: "f70cd1d5-cb51-496c-a2d2-face4d7af1a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:13:24.784262 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.784242 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-home\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:24.784262 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.784256 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-model-cache\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:24.786264 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.786225 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-kube-api-access-nwvdz" (OuterVolumeSpecName: "kube-api-access-nwvdz") pod "f70cd1d5-cb51-496c-a2d2-face4d7af1a5" (UID: "f70cd1d5-cb51-496c-a2d2-face4d7af1a5"). InnerVolumeSpecName "kube-api-access-nwvdz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 22:13:24.786389 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.786300 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f70cd1d5-cb51-496c-a2d2-face4d7af1a5" (UID: "f70cd1d5-cb51-496c-a2d2-face4d7af1a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 22:13:24.786389 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.786306 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-dshm" (OuterVolumeSpecName: "dshm") pod "f70cd1d5-cb51-496c-a2d2-face4d7af1a5" (UID: "f70cd1d5-cb51-496c-a2d2-face4d7af1a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:13:24.872978 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.872936 2554 generic.go:356] "Generic (PLEG): container finished" podID="f70cd1d5-cb51-496c-a2d2-face4d7af1a5" containerID="b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147" exitCode=0 Jul 17 22:13:24.872978 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.872976 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" Jul 17 22:13:24.873242 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.872987 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" event={"ID":"f70cd1d5-cb51-496c-a2d2-face4d7af1a5","Type":"ContainerDied","Data":"b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147"} Jul 17 22:13:24.873242 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.873038 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg" event={"ID":"f70cd1d5-cb51-496c-a2d2-face4d7af1a5","Type":"ContainerDied","Data":"7f2d5f8abfde56f6b1e8caa246d0772f4f3a7642238e337b0c1101ecc781b30a"} Jul 17 22:13:24.873242 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.873060 2554 scope.go:117] "RemoveContainer" containerID="b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147" Jul 17 22:13:24.882691 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.882670 2554 scope.go:117] "RemoveContainer" containerID="b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147" Jul 17 22:13:24.882965 ip-10-0-131-188 kubenswrapper[2554]: E0717 22:13:24.882945 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147\": container with ID starting with b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147 not found: ID does not exist" containerID="b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147" Jul 17 22:13:24.883029 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.882974 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147"} err="failed to get container status \"b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147\": rpc error: code = NotFound desc = could not find container \"b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147\": container with ID starting with b6ced243fcd5f9858646f716f81a939f22c58c9a212e6a9960b7192b68886147 not found: ID does not exist" Jul 17 22:13:24.885611 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.885591 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-dshm\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:24.885711 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.885618 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwvdz\" (UniqueName: \"kubernetes.io/projected/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-kube-api-access-nwvdz\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:24.885711 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.885632 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tls-certs\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:24.885711 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.885644 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70cd1d5-cb51-496c-a2d2-face4d7af1a5-tmp-dir\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:24.895709 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.895653 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg"] Jul 17 22:13:24.900743 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:24.900720 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-2a645a/rate-test-model-2a645a-kserve-77d6d94f7f-qjxbg"] Jul 17 22:13:25.243977 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.243864 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" podUID="9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" containerName="main" containerID="cri-o://3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f" gracePeriod=45 Jul 17 22:13:25.500189 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.500125 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:13:25.691517 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.691475 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-dshm\") pod \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " Jul 17 22:13:25.691517 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.691525 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-model-cache\") pod \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " Jul 17 22:13:25.691831 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.691588 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7k5qd\" (UniqueName: \"kubernetes.io/projected/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-kube-api-access-7k5qd\") pod \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " Jul 17 22:13:25.691831 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.691655 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tmp-dir\") pod \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " Jul 17 22:13:25.691831 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.691674 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tls-certs\") pod \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " Jul 17 22:13:25.691831 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.691701 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-home\") pod \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\" (UID: \"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4\") " Jul 17 22:13:25.692052 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.691871 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-model-cache" (OuterVolumeSpecName: "model-cache") pod "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" (UID: "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:13:25.692052 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.691902 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" (UID: "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:13:25.692052 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.692008 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-home" (OuterVolumeSpecName: "home") pod "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" (UID: "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:13:25.692210 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.692069 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tmp-dir\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:25.692210 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.692082 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-home\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:25.692210 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.692090 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-model-cache\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:25.694080 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.694054 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-dshm" (OuterVolumeSpecName: "dshm") pod "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" (UID: "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 22:13:25.694205 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.694078 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-kube-api-access-7k5qd" (OuterVolumeSpecName: "kube-api-access-7k5qd") pod "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" (UID: "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4"). InnerVolumeSpecName "kube-api-access-7k5qd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 22:13:25.694205 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.694055 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" (UID: "9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 22:13:25.721153 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.721118 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f70cd1d5-cb51-496c-a2d2-face4d7af1a5" path="/var/lib/kubelet/pods/f70cd1d5-cb51-496c-a2d2-face4d7af1a5/volumes" Jul 17 22:13:25.793207 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.793172 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-dshm\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:25.793207 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.793203 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k5qd\" (UniqueName: \"kubernetes.io/projected/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-kube-api-access-7k5qd\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:25.793207 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.793213 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4-tls-certs\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:25.878910 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.878877 2554 generic.go:356] "Generic (PLEG): container finished" podID="9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" containerID="3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f" exitCode=0 Jul 17 22:13:25.879126 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.878915 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" Jul 17 22:13:25.879126 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.878965 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" event={"ID":"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4","Type":"ContainerDied","Data":"3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f"} Jul 17 22:13:25.879126 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.879009 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n" event={"ID":"9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4","Type":"ContainerDied","Data":"7ffae6a9d27decf0708e09e20c4da69319029968ba5e27bab8876f7ab74f7f02"} Jul 17 22:13:25.879126 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.879031 2554 scope.go:117] "RemoveContainer" containerID="3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f" Jul 17 22:13:25.888215 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.888192 2554 scope.go:117] "RemoveContainer" containerID="3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f" Jul 17 22:13:25.888493 ip-10-0-131-188 kubenswrapper[2554]: E0717 22:13:25.888476 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f\": container with ID starting with 3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f not found: ID does not exist" containerID="3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f" Jul 17 22:13:25.888574 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.888503 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f"} err="failed to get container status \"3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f\": rpc error: code = NotFound desc = could not find container \"3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f\": container with ID starting with 3257d2bb296ef9e8da873e3880347f2fb5fe2e5d1ceed2717200bea0195e646f not found: ID does not exist" Jul 17 22:13:25.896896 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.896872 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n"] Jul 17 22:13:25.901151 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:25.901128 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b96f25/rate-test-model-b96f25-kserve-77c977f679-zhg9n"] Jul 17 22:13:26.075022 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.074939 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7513-139902992824624"] Jul 17 22:13:26.075385 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.075370 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" containerName="main" Jul 17 22:13:26.075426 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.075388 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" containerName="main" Jul 17 22:13:26.075426 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.075397 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f70cd1d5-cb51-496c-a2d2-face4d7af1a5" containerName="main" Jul 17 22:13:26.075426 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.075403 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f70cd1d5-cb51-496c-a2d2-face4d7af1a5" containerName="main" Jul 17 22:13:26.075515 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.075483 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f70cd1d5-cb51-496c-a2d2-face4d7af1a5" containerName="main" Jul 17 22:13:26.075515 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.075493 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" containerName="main" Jul 17 22:13:26.080297 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.080270 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7513-139902992824624" Jul 17 22:13:26.082914 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.082867 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-s7gsx\"" Jul 17 22:13:26.083305 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.083280 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7513-139902992824624"] Jul 17 22:13:26.097134 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.097111 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srwsp\" (UniqueName: \"kubernetes.io/projected/63b1ddd6-ef14-4d64-90a7-80b8d54325ed-kube-api-access-srwsp\") pod \"test-curl-7513-139902992824624\" (UID: \"63b1ddd6-ef14-4d64-90a7-80b8d54325ed\") " pod="opendatahub/test-curl-7513-139902992824624" Jul 17 22:13:26.198709 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.198675 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srwsp\" (UniqueName: \"kubernetes.io/projected/63b1ddd6-ef14-4d64-90a7-80b8d54325ed-kube-api-access-srwsp\") pod \"test-curl-7513-139902992824624\" (UID: \"63b1ddd6-ef14-4d64-90a7-80b8d54325ed\") " pod="opendatahub/test-curl-7513-139902992824624" Jul 17 22:13:26.207385 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.207360 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srwsp\" (UniqueName: \"kubernetes.io/projected/63b1ddd6-ef14-4d64-90a7-80b8d54325ed-kube-api-access-srwsp\") pod \"test-curl-7513-139902992824624\" (UID: \"63b1ddd6-ef14-4d64-90a7-80b8d54325ed\") " pod="opendatahub/test-curl-7513-139902992824624" Jul 17 22:13:26.392120 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.392037 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7513-139902992824624" Jul 17 22:13:26.535175 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.535149 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7513-139902992824624"] Jul 17 22:13:26.537183 ip-10-0-131-188 kubenswrapper[2554]: W0717 22:13:26.537153 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63b1ddd6_ef14_4d64_90a7_80b8d54325ed.slice/crio-67f5a3050e4834aef2d5ed46e876e4688340b6d9a630ab40ca9435cfc78b0263 WatchSource:0}: Error finding container 67f5a3050e4834aef2d5ed46e876e4688340b6d9a630ab40ca9435cfc78b0263: Status 404 returned error can't find the container with id 67f5a3050e4834aef2d5ed46e876e4688340b6d9a630ab40ca9435cfc78b0263 Jul 17 22:13:26.538468 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.538450 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 22:13:26.890198 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:26.890121 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7513-139902992824624" event={"ID":"63b1ddd6-ef14-4d64-90a7-80b8d54325ed","Type":"ContainerStarted","Data":"67f5a3050e4834aef2d5ed46e876e4688340b6d9a630ab40ca9435cfc78b0263"} Jul 17 22:13:27.725323 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:27.725288 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4" path="/var/lib/kubelet/pods/9bea0f52-7a00-4a27-b1a9-b8cc9218ccd4/volumes" Jul 17 22:13:27.896607 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:27.896569 2554 generic.go:356] "Generic (PLEG): container finished" podID="63b1ddd6-ef14-4d64-90a7-80b8d54325ed" containerID="820991104a3e0b4723312e9f43c48ab48c2245f8086c9d09c1e7c34bd16622ef" exitCode=0 Jul 17 22:13:27.896797 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:27.896645 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7513-139902992824624" event={"ID":"63b1ddd6-ef14-4d64-90a7-80b8d54325ed","Type":"ContainerDied","Data":"820991104a3e0b4723312e9f43c48ab48c2245f8086c9d09c1e7c34bd16622ef"} Jul 17 22:13:29.045737 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.045711 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7513-139902992824624" Jul 17 22:13:29.068181 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.068159 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7513-139902992824624_63b1ddd6-ef14-4d64-90a7-80b8d54325ed/test-curl-7513-139902992824624/0.log" Jul 17 22:13:29.089748 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.089721 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7513-139902992824624"] Jul 17 22:13:29.093783 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.093760 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7513-139902992824624"] Jul 17 22:13:29.127782 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.127743 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srwsp\" (UniqueName: \"kubernetes.io/projected/63b1ddd6-ef14-4d64-90a7-80b8d54325ed-kube-api-access-srwsp\") pod \"63b1ddd6-ef14-4d64-90a7-80b8d54325ed\" (UID: \"63b1ddd6-ef14-4d64-90a7-80b8d54325ed\") " Jul 17 22:13:29.130019 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.129999 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63b1ddd6-ef14-4d64-90a7-80b8d54325ed-kube-api-access-srwsp" (OuterVolumeSpecName: "kube-api-access-srwsp") pod "63b1ddd6-ef14-4d64-90a7-80b8d54325ed" (UID: "63b1ddd6-ef14-4d64-90a7-80b8d54325ed"). InnerVolumeSpecName "kube-api-access-srwsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 22:13:29.229313 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.229229 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srwsp\" (UniqueName: \"kubernetes.io/projected/63b1ddd6-ef14-4d64-90a7-80b8d54325ed-kube-api-access-srwsp\") on node \"ip-10-0-131-188.ec2.internal\" DevicePath \"\"" Jul 17 22:13:29.721609 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.721569 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63b1ddd6-ef14-4d64-90a7-80b8d54325ed" path="/var/lib/kubelet/pods/63b1ddd6-ef14-4d64-90a7-80b8d54325ed/volumes" Jul 17 22:13:29.795410 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.795372 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:29.868323 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.868291 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:29.908174 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.908146 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7513-139902992824624" Jul 17 22:13:29.908365 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:29.908189 2554 scope.go:117] "RemoveContainer" containerID="820991104a3e0b4723312e9f43c48ab48c2245f8086c9d09c1e7c34bd16622ef" Jul 17 22:13:39.710101 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:39.710069 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:43.036112 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:43.036069 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:13:44.362351 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:44.362318 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:45.260694 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:45.260662 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:45.361963 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:45.361930 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:50.345388 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:50.345344 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:50.455362 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:50.455273 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:55.036949 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:55.036912 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:55.264264 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:55.264231 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:56.061606 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:56.061572 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:13:56.165860 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:13:56.165828 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:14:05.834287 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:05.834246 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:14:09.336407 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:09.336371 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vksmg"] Jul 17 22:14:10.470328 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:10.470287 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:14:16.447107 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:16.447068 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:14:16.513208 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:16.513171 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:14:21.144927 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:21.144896 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:14:21.372572 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:21.372518 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:14:26.865333 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:26.865296 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xnlqd_b5b821f5-5bd8-423d-b0c7-bbcebc1832d9/manager/0.log" Jul 17 22:14:26.980238 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:26.980203 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dzl9s_0f19398a-d921-4131-85d9-a0245a13d4c2/manager/0.log" Jul 17 22:14:27.204212 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:27.204125 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dvw4f_410071d2-6321-4391-9165-39499b01b2d8/server/0.log" Jul 17 22:14:27.313028 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:27.312983 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c7kp5_daabaacb-517b-4583-b831-4ee73d8dc255/manager/0.log" Jul 17 22:14:27.679078 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:27.679054 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6dbd9688-zk9jw_6c3c450c-8ea9-4253-8733-ab4ca75cc368/copy-manifests/0.log" Jul 17 22:14:27.685927 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:27.685900 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6dbd9688-zk9jw_6c3c450c-8ea9-4253-8733-ab4ca75cc368/manager/0.log" Jul 17 22:14:30.758585 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:30.758539 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mzfnm_91a70b87-46f9-4d22-af69-07bf8efef718/manager/0.log" Jul 17 22:14:30.864422 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:30.864394 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5srfx_db5c16d5-a277-41eb-a3ed-4df6f7bb9330/manager/0.log" Jul 17 22:14:30.970257 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:30.970230 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-n9l8f_8832db76-f54a-4b86-a1bb-e869385310b2/kuadrant-console-plugin/0.log" Jul 17 22:14:31.080621 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:31.080589 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-t2z5f_5ba6a147-13f4-467b-966d-faeb5482199f/registry-server/0.log" Jul 17 22:14:31.194739 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:31.194710 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2x66q_c1b01572-4ec7-4607-9aa4-b894b1de2277/manager/0.log" Jul 17 22:14:31.307202 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:31.307175 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-vksmg_1d99b42c-a11c-4469-8ce9-db57c07cf719/limitador/0.log" Jul 17 22:14:31.766298 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:31.766266 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qq65x_a89ca970-488e-40e2-b7ad-660b941a47c5/istio-proxy/0.log" Jul 17 22:14:32.087310 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:32.087282 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fbb8dc498-xsnxw_62f4330c-55ea-46a3-9027-f1723405db8e/kube-auth-proxy/0.log" Jul 17 22:14:32.204692 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:32.204666 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gkjn5_2c68926a-c530-4f68-ab8c-4e34730f5664/istio-proxy/1.log" Jul 17 22:14:32.542975 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:32.542926 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74bdb64d9b-rxs6z_133e55a6-f3de-428f-a9c6-fa697726d63a/router/0.log" Jul 17 22:14:33.079412 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:33.079383 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vlkpn_745ea714-346c-467b-88ca-d1305f634685/main/0.log" Jul 17 22:14:33.298143 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:33.298115 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-xjlc9_72e3b4a6-08ae-4936-ad1d-a4e8afa67de9/main/0.log" Jul 17 22:14:33.410297 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:33.410213 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-7dc8b_25c88dd9-90d6-4e3a-b06a-6142537d3c39/main/0.log" Jul 17 22:14:38.416455 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:38.416426 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:14:38.516997 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:38.516962 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" limit=1024 Jul 17 22:14:41.193495 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:41.193460 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-79tgn_6964db66-5d84-43e3-8f76-e7e23aeb63d3/global-pull-secret-syncer/0.log" Jul 17 22:14:41.279390 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:41.279359 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-grpkq_f4eb2215-40df-4faf-9040-0b4a38ed8198/konnectivity-agent/0.log" Jul 17 22:14:41.398253 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:41.398224 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-188.ec2.internal_fba93f9b97932f873e087f216da95d10/haproxy/0.log" Jul 17 22:14:46.944456 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:46.944422 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mzfnm_91a70b87-46f9-4d22-af69-07bf8efef718/manager/0.log" Jul 17 22:14:46.970717 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:46.970685 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5srfx_db5c16d5-a277-41eb-a3ed-4df6f7bb9330/manager/0.log" Jul 17 22:14:46.997320 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:46.997296 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-n9l8f_8832db76-f54a-4b86-a1bb-e869385310b2/kuadrant-console-plugin/0.log" Jul 17 22:14:47.053063 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:47.053021 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-t2z5f_5ba6a147-13f4-467b-966d-faeb5482199f/registry-server/0.log" Jul 17 22:14:47.182329 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:47.182295 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2x66q_c1b01572-4ec7-4607-9aa4-b894b1de2277/manager/0.log" Jul 17 22:14:47.239434 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:47.239356 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-vksmg_1d99b42c-a11c-4469-8ce9-db57c07cf719/limitador/0.log" Jul 17 22:14:49.161689 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:49.161655 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-94h2v_a491e347-ad20-4da2-97bb-ce0d561846bb/cluster-monitoring-operator/0.log" Jul 17 22:14:49.487048 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:49.486968 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tc4x7_9218f58d-90fe-42fe-a695-60cee3beb9ca/node-exporter/0.log" Jul 17 22:14:49.506767 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:49.506741 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tc4x7_9218f58d-90fe-42fe-a695-60cee3beb9ca/kube-rbac-proxy/0.log" Jul 17 22:14:49.530085 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:49.530058 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tc4x7_9218f58d-90fe-42fe-a695-60cee3beb9ca/init-textfile/0.log" Jul 17 22:14:51.173861 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.173823 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd"] Jul 17 22:14:51.174386 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.174366 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63b1ddd6-ef14-4d64-90a7-80b8d54325ed" containerName="test-curl-7513-139902992824624" Jul 17 22:14:51.174467 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.174390 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="63b1ddd6-ef14-4d64-90a7-80b8d54325ed" containerName="test-curl-7513-139902992824624" Jul 17 22:14:51.174597 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.174583 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="63b1ddd6-ef14-4d64-90a7-80b8d54325ed" containerName="test-curl-7513-139902992824624" Jul 17 22:14:51.177731 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.177710 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.180217 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.180198 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gjx6d\"/\"kube-root-ca.crt\"" Jul 17 22:14:51.181247 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.181230 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gjx6d\"/\"openshift-service-ca.crt\"" Jul 17 22:14:51.181365 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.181245 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gjx6d\"/\"default-dockercfg-sgfdh\"" Jul 17 22:14:51.184339 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.184321 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd"] Jul 17 22:14:51.262028 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.261996 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-vt6tr_dc2d76be-257f-4a0e-9e8b-45ed8cf0ea71/networking-console-plugin/0.log" Jul 17 22:14:51.279186 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.279155 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-podres\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.279379 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.279195 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-lib-modules\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.279379 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.279213 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-sys\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.279379 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.279349 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbd64\" (UniqueName: \"kubernetes.io/projected/1cefffd9-90cf-481f-8fcd-052dabbe7bce-kube-api-access-tbd64\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.279540 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.279435 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-proc\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.380847 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.380805 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-proc\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.381045 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.380883 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-podres\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.381045 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.380912 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-lib-modules\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.381045 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.380931 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-sys\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.381045 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.380944 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-proc\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.381045 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.381002 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-sys\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.381045 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.381022 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbd64\" (UniqueName: \"kubernetes.io/projected/1cefffd9-90cf-481f-8fcd-052dabbe7bce-kube-api-access-tbd64\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.381272 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.381059 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-podres\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.381272 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.381084 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cefffd9-90cf-481f-8fcd-052dabbe7bce-lib-modules\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.389773 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.389746 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbd64\" (UniqueName: \"kubernetes.io/projected/1cefffd9-90cf-481f-8fcd-052dabbe7bce-kube-api-access-tbd64\") pod \"perf-node-gather-daemonset-vjpsd\" (UID: \"1cefffd9-90cf-481f-8fcd-052dabbe7bce\") " pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.488409 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.488322 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:51.641599 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.641561 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd"] Jul 17 22:14:51.644056 ip-10-0-131-188 kubenswrapper[2554]: W0717 22:14:51.644026 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1cefffd9_90cf_481f_8fcd_052dabbe7bce.slice/crio-3d5b282b78a66788831d73b84b41fc5faa17aa7429d5f9f46088d20df410a464 WatchSource:0}: Error finding container 3d5b282b78a66788831d73b84b41fc5faa17aa7429d5f9f46088d20df410a464: Status 404 returned error can't find the container with id 3d5b282b78a66788831d73b84b41fc5faa17aa7429d5f9f46088d20df410a464 Jul 17 22:14:51.865119 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.865029 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/2.log" Jul 17 22:14:51.869526 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:51.869502 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qdvms_9cefc4af-8f20-4aad-ba30-041f244f9740/console-operator/3.log" Jul 17 22:14:52.317895 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:52.317860 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" event={"ID":"1cefffd9-90cf-481f-8fcd-052dabbe7bce","Type":"ContainerStarted","Data":"18bb816425afbffd54737b11183999c1048db1172426aae52adea95cb439e859"} Jul 17 22:14:52.318310 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:52.317917 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" event={"ID":"1cefffd9-90cf-481f-8fcd-052dabbe7bce","Type":"ContainerStarted","Data":"3d5b282b78a66788831d73b84b41fc5faa17aa7429d5f9f46088d20df410a464"} Jul 17 22:14:52.318310 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:52.317943 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:52.335234 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:52.335177 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" podStartSLOduration=1.335162161 podStartE2EDuration="1.335162161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 22:14:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 22:14:52.333624289 +0000 UTC m=+3473.102972269" watchObservedRunningTime="2026-07-17 22:14:52.335162161 +0000 UTC m=+3473.104510139" Jul 17 22:14:52.916253 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:52.916224 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-ppljx_389d5d02-d0be-4eff-bdea-bb0c695ca377/volume-data-source-validator/0.log" Jul 17 22:14:53.973213 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:53.973180 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rvqhc_17c76d2b-6ab9-421d-97af-7c9e03181c7c/dns/0.log" Jul 17 22:14:53.998782 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:53.998751 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rvqhc_17c76d2b-6ab9-421d-97af-7c9e03181c7c/kube-rbac-proxy/0.log" Jul 17 22:14:54.120663 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:54.120637 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rjxhs_768859c6-079b-4947-ad85-d6bf78ac1023/dns-node-resolver/0.log" Jul 17 22:14:54.676803 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:54.676776 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6zztr_82b932c6-788d-4a01-bdcb-c8c9b7303984/node-ca/0.log" Jul 17 22:14:55.651265 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:55.651233 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qq65x_a89ca970-488e-40e2-b7ad-660b941a47c5/istio-proxy/0.log" Jul 17 22:14:55.811341 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:55.811311 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fbb8dc498-xsnxw_62f4330c-55ea-46a3-9027-f1723405db8e/kube-auth-proxy/0.log" Jul 17 22:14:55.840339 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:55.840310 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gkjn5_2c68926a-c530-4f68-ab8c-4e34730f5664/istio-proxy/0.log" Jul 17 22:14:55.869817 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:55.869786 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gkjn5_2c68926a-c530-4f68-ab8c-4e34730f5664/istio-proxy/1.log" Jul 17 22:14:55.954647 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:55.954566 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74bdb64d9b-rxs6z_133e55a6-f3de-428f-a9c6-fa697726d63a/router/0.log" Jul 17 22:14:56.660613 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:56.660574 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hfktc_e1f9a694-e7b7-4222-907d-583429aa5cf3/serve-healthcheck-canary/0.log" Jul 17 22:14:57.179527 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:57.179498 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-k9wjf_999ecc4f-66b1-4ecb-97c5-c54040757d19/insights-operator/0.log" Jul 17 22:14:57.180607 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:57.180593 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-k9wjf_999ecc4f-66b1-4ecb-97c5-c54040757d19/insights-operator/1.log" Jul 17 22:14:57.201622 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:57.201599 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxrcv_fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f/kube-rbac-proxy/0.log" Jul 17 22:14:57.223522 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:57.223498 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxrcv_fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f/exporter/0.log" Jul 17 22:14:57.245624 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:57.245601 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxrcv_fa3b9fc0-328c-44c6-8efd-2c5a5bf6c70f/extractor/0.log" Jul 17 22:14:58.333614 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:58.333578 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gjx6d/perf-node-gather-daemonset-vjpsd" Jul 17 22:14:59.476203 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:59.476166 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xnlqd_b5b821f5-5bd8-423d-b0c7-bbcebc1832d9/manager/0.log" Jul 17 22:14:59.619452 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:59.619422 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dzl9s_0f19398a-d921-4131-85d9-a0245a13d4c2/manager/0.log" Jul 17 22:14:59.791862 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:59.791837 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dvw4f_410071d2-6321-4391-9165-39499b01b2d8/server/0.log" Jul 17 22:14:59.863652 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:14:59.863622 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c7kp5_daabaacb-517b-4583-b831-4ee73d8dc255/manager/0.log" Jul 17 22:15:00.085322 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:00.085295 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6dbd9688-zk9jw_6c3c450c-8ea9-4253-8733-ab4ca75cc368/manager/0.log" Jul 17 22:15:00.137774 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:00.137747 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6dbd9688-zk9jw_6c3c450c-8ea9-4253-8733-ab4ca75cc368/copy-manifests/0.log" Jul 17 22:15:02.267775 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:02.267743 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qq65x_a89ca970-488e-40e2-b7ad-660b941a47c5/istio-proxy/0.log" Jul 17 22:15:02.414121 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:02.414094 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fbb8dc498-xsnxw_62f4330c-55ea-46a3-9027-f1723405db8e/kube-auth-proxy/0.log" Jul 17 22:15:02.431508 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:02.431434 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gkjn5_2c68926a-c530-4f68-ab8c-4e34730f5664/istio-proxy/0.log" Jul 17 22:15:02.463261 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:02.463231 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gkjn5_2c68926a-c530-4f68-ab8c-4e34730f5664/istio-proxy/1.log" Jul 17 22:15:02.535834 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:02.535808 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74bdb64d9b-rxs6z_133e55a6-f3de-428f-a9c6-fa697726d63a/router/0.log" Jul 17 22:15:03.664620 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:03.664591 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qq65x_a89ca970-488e-40e2-b7ad-660b941a47c5/istio-proxy/0.log" Jul 17 22:15:03.799897 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:03.799845 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fbb8dc498-xsnxw_62f4330c-55ea-46a3-9027-f1723405db8e/kube-auth-proxy/0.log" Jul 17 22:15:03.819674 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:03.819644 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gkjn5_2c68926a-c530-4f68-ab8c-4e34730f5664/istio-proxy/0.log" Jul 17 22:15:03.850221 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:03.850190 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gkjn5_2c68926a-c530-4f68-ab8c-4e34730f5664/istio-proxy/1.log" Jul 17 22:15:03.922854 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:03.922778 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74bdb64d9b-rxs6z_133e55a6-f3de-428f-a9c6-fa697726d63a/router/0.log" Jul 17 22:15:06.986064 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:06.986028 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-w8cbj_036870c8-94b9-4906-8b4a-ea70695c9c79/kube-storage-version-migrator-operator/1.log" Jul 17 22:15:06.986802 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:06.986782 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-w8cbj_036870c8-94b9-4906-8b4a-ea70695c9c79/kube-storage-version-migrator-operator/0.log" Jul 17 22:15:07.935537 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:07.935505 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-57924_dbdbe4e0-e2fb-4591-93b3-6eaf6d7b9c8a/kube-multus/0.log" Jul 17 22:15:08.308532 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:08.308505 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkkg4_6b870da3-94e0-4301-8be9-17174f8a012a/kube-multus-additional-cni-plugins/0.log" Jul 17 22:15:08.329755 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:08.329724 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkkg4_6b870da3-94e0-4301-8be9-17174f8a012a/egress-router-binary-copy/0.log" Jul 17 22:15:08.350945 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:08.350920 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkkg4_6b870da3-94e0-4301-8be9-17174f8a012a/cni-plugins/0.log" Jul 17 22:15:08.375186 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:08.375156 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkkg4_6b870da3-94e0-4301-8be9-17174f8a012a/bond-cni-plugin/0.log" Jul 17 22:15:08.395347 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:08.395319 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkkg4_6b870da3-94e0-4301-8be9-17174f8a012a/routeoverride-cni/0.log" Jul 17 22:15:08.415757 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:08.415727 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkkg4_6b870da3-94e0-4301-8be9-17174f8a012a/whereabouts-cni-bincopy/0.log" Jul 17 22:15:08.434097 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:08.434075 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkkg4_6b870da3-94e0-4301-8be9-17174f8a012a/whereabouts-cni/0.log" Jul 17 22:15:08.488729 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:08.488699 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6w8mk_7507ea3a-57dc-4701-aaf0-5e241ab82b74/network-metrics-daemon/0.log" Jul 17 22:15:08.506159 ip-10-0-131-188 kubenswrapper[2554]: I0717 22:15:08.506131 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6w8mk_7507ea3a-57dc-4701-aaf0-5e241ab82b74/kube-rbac-proxy/0.log"