Jul 20 12:42:36.675755 ip-10-0-141-229 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 20 12:42:36.675770 ip-10-0-141-229 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 20 12:42:36.675779 ip-10-0-141-229 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 20 12:42:36.676103 ip-10-0-141-229 systemd[1]: Failed to start Kubernetes Kubelet. Jul 20 12:42:46.793698 ip-10-0-141-229 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 20 12:42:46.793714 ip-10-0-141-229 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b7b9e778232d4768ba9d9d492552ab20 -- Jul 20 12:44:17.896711 ip-10-0-141-229 systemd[1]: Starting Kubernetes Kubelet... Jul 20 12:44:18.346691 ip-10-0-141-229 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 12:44:18.346691 ip-10-0-141-229 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 12:44:18.346691 ip-10-0-141-229 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 12:44:18.346691 ip-10-0-141-229 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 12:44:18.346691 ip-10-0-141-229 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 12:44:18.347391 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.346687 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 12:44:18.354577 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354548 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 12:44:18.354577 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354573 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 12:44:18.354577 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354577 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 12:44:18.354577 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354580 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 12:44:18.354577 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354583 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354586 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354591 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354594 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354596 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354599 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354602 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354605 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354607 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354611 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354614 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354617 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354619 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354622 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354624 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354627 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354629 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354632 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354635 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354637 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 12:44:18.354735 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354640 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354642 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354646 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354648 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354652 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354654 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354657 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354659 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354662 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354664 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354667 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354669 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354672 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354675 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354679 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354681 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354684 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354687 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354689 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354692 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 12:44:18.355215 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354694 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354697 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354700 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354702 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354705 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354708 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354710 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354712 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354715 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354717 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354720 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354722 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354724 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354727 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354729 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354732 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354734 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354736 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354741 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 12:44:18.355732 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354745 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354750 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354753 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354757 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354761 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354763 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354766 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354769 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354772 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354774 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354777 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354780 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354782 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354785 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354787 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354791 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354795 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354800 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354803 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 12:44:18.356204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354806 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354809 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354811 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354814 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354817 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354819 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354821 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354824 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354826 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354829 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354831 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354834 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354837 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354840 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354842 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354845 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.354848 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356782 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356800 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356805 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 12:44:18.356902 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356809 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356815 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356820 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356823 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356828 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356832 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356836 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356840 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356844 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356849 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356853 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356857 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356861 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356864 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356869 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356874 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356878 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356882 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356886 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356889 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 12:44:18.357414 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356895 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356899 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356904 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356910 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356917 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356922 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356927 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356931 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356934 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356936 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356939 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356942 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356945 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356947 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356951 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356954 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356957 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356959 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356962 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 12:44:18.357926 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356965 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356967 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356970 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356972 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356975 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356977 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356980 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356983 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356986 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356990 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356994 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.356997 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357000 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357003 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357005 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357008 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357011 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357014 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357016 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 12:44:18.358391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357019 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357022 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357024 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357027 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357029 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357032 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357034 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357037 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357040 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357042 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357044 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357047 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357051 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357053 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357056 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357058 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357061 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357064 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357067 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357069 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 12:44:18.358867 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357071 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357074 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357078 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357081 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357084 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357086 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357089 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357091 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357094 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357098 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357101 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357104 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357107 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357110 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357112 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357115 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357117 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357120 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357533 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357539 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 12:44:18.359348 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357543 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357546 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357548 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357551 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357554 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357557 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357559 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357562 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357564 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357567 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357569 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357572 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357574 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357577 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357581 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357584 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357587 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357590 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357593 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 12:44:18.359888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357595 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357598 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357601 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357604 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357606 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357609 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357611 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357614 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357617 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357620 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357622 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357625 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357628 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357631 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357633 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357636 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357639 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357641 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357644 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357646 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357649 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 12:44:18.360341 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357652 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357654 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357657 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357660 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357663 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357666 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357669 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357672 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357675 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357677 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357680 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357683 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357685 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357688 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357690 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357692 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357695 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357697 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357700 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357703 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 12:44:18.360860 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357706 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357708 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357711 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357715 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357718 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357720 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357723 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357725 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357728 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357730 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357733 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357735 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357740 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357743 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357746 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357750 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357753 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357758 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357764 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 12:44:18.361392 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357767 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357770 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357773 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357775 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357778 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357781 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357783 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357786 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357789 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357791 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357794 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357796 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357799 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357802 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357813 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357817 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357820 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.357822 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357896 2570 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357908 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 12:44:18.361888 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357944 2570 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357949 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357954 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357958 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357963 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357967 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357970 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357973 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357977 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357980 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357985 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357989 2570 flags.go:64] FLAG: --cgroup-root="" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357992 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357995 2570 flags.go:64] FLAG: --client-ca-file="" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.357999 2570 flags.go:64] FLAG: --cloud-provider="external" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358002 2570 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358007 2570 flags.go:64] FLAG: --cluster-domain="" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358010 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358013 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358017 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358020 2570 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358024 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358027 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 12:44:18.362429 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358030 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358034 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358037 2570 flags.go:64] FLAG: --contention-profiling="false" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358040 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358044 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358047 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358050 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358054 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358057 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358060 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358063 2570 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358066 2570 flags.go:64] FLAG: --enable-server="true" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358069 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358073 2570 flags.go:64] FLAG: --event-burst="100" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358076 2570 flags.go:64] FLAG: --event-qps="50" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358079 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358082 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358085 2570 flags.go:64] FLAG: --eviction-hard="" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358089 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358093 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358097 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358100 2570 flags.go:64] FLAG: --eviction-soft="" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358103 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358106 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358110 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 12:44:18.362993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358113 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358116 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358119 2570 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358122 2570 flags.go:64] FLAG: --feature-gates="" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358126 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358129 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358132 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358135 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358138 2570 flags.go:64] FLAG: --healthz-port="10248" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358141 2570 flags.go:64] FLAG: --help="false" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358144 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358147 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358150 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358153 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358156 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358159 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358162 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358165 2570 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358168 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358171 2570 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358174 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358177 2570 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358179 2570 flags.go:64] FLAG: --kube-reserved="" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358182 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 12:44:18.363627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358185 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358188 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358192 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358196 2570 flags.go:64] FLAG: --lock-file="" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358199 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358202 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358205 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358211 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358215 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358217 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358220 2570 flags.go:64] FLAG: --logging-format="text" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358223 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358226 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358229 2570 flags.go:64] FLAG: --manifest-url="" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358232 2570 flags.go:64] FLAG: --manifest-url-header="" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358236 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358239 2570 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358243 2570 flags.go:64] FLAG: --max-pods="110" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358246 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358249 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358251 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358254 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358257 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358260 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358263 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 12:44:18.364214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358270 2570 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358273 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358277 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358280 2570 flags.go:64] FLAG: --pod-cidr="" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358283 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358288 2570 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358291 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358294 2570 flags.go:64] FLAG: --pods-per-core="0" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358297 2570 flags.go:64] FLAG: --port="10250" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358301 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358304 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07553359707a74b4d" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358308 2570 flags.go:64] FLAG: --qos-reserved="" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358312 2570 flags.go:64] FLAG: --read-only-port="10255" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358315 2570 flags.go:64] FLAG: --register-node="true" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358318 2570 flags.go:64] FLAG: --register-with-taints="" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358321 2570 flags.go:64] FLAG: --registry-burst="10" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358325 2570 flags.go:64] FLAG: --registry-qps="5" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358328 2570 flags.go:64] FLAG: --reserved-cpus="" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358331 2570 flags.go:64] FLAG: --reserved-memory="" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358340 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358343 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358346 2570 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358348 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358351 2570 flags.go:64] FLAG: --runonce="false" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358354 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 12:44:18.364865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358358 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358361 2570 flags.go:64] FLAG: --seccomp-default="false" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358378 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358382 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358385 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358388 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358391 2570 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358395 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358398 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358401 2570 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358403 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358406 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358409 2570 flags.go:64] FLAG: --system-cgroups="" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358412 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358418 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358421 2570 flags.go:64] FLAG: --tls-cert-file="" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358424 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358429 2570 flags.go:64] FLAG: --tls-min-version="" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358433 2570 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358436 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358439 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358442 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358445 2570 flags.go:64] FLAG: --v="2" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358449 2570 flags.go:64] FLAG: --version="false" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358454 2570 flags.go:64] FLAG: --vmodule="" Jul 20 12:44:18.365514 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358458 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.358461 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358579 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358584 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358587 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358590 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358593 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358596 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358599 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358602 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358605 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358608 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358611 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358613 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358617 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358619 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358622 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358625 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358628 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358630 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 12:44:18.366204 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358633 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358635 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358637 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358640 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358643 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358647 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358649 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358652 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358654 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358657 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358659 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358662 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358665 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358668 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358670 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358673 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358676 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358678 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358681 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 12:44:18.366729 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358683 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358686 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358690 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358693 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358697 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358700 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358702 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358705 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358708 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358712 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358714 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358717 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358720 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358723 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358727 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358730 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358732 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358735 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358739 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 12:44:18.367189 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358742 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358744 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358747 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358749 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358752 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358755 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358758 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358761 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358763 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358766 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358768 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358771 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358773 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358776 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358778 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358781 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358783 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358786 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358788 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358791 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 12:44:18.367684 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358794 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358796 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358798 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358801 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358803 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358806 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358809 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358811 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358814 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358816 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358819 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358822 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358825 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358827 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358830 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358833 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358835 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358838 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358841 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 12:44:18.368181 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358843 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358847 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358849 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.358852 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.359487 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.367002 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.367023 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.367043 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.367052 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367113 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367119 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367123 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367126 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367129 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367132 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 12:44:18.368767 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367134 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367137 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367140 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367142 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367145 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367147 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367150 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367153 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367155 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367158 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367160 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367163 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367166 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367169 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367171 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367174 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367176 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367179 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367182 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367185 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 12:44:18.369142 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367188 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367190 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367193 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367198 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367201 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367204 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367207 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367209 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367213 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367218 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367220 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367223 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367226 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367228 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367231 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367234 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367236 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367239 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367241 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367244 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 12:44:18.369652 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367246 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367249 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367251 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367254 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367257 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367259 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367262 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367266 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367268 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367271 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367274 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367277 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367279 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367282 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367285 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367288 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367291 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367295 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367297 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 12:44:18.370158 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367300 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367302 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367306 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367310 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367313 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367316 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367319 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367323 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367326 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367329 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367332 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367335 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367338 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367342 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367345 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367348 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367351 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367353 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367356 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 12:44:18.370634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367359 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367361 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367378 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367383 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367386 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367389 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367392 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367395 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367398 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367401 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367404 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367407 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367411 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367414 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367416 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 12:44:18.371092 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.367422 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367548 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367554 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367556 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367559 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367562 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367565 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367567 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367570 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367572 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367575 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367577 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367580 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367582 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367585 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367588 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367590 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367593 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367596 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367598 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367601 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 12:44:18.371488 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367603 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367606 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367609 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367612 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367615 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367617 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367620 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367623 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367626 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367629 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367632 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367637 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367639 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367642 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367645 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367648 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367650 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367654 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367657 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 12:44:18.372040 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367660 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367663 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367665 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367668 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367671 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367674 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367677 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367681 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367684 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367686 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367689 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367692 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367694 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367697 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367700 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367703 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367706 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367708 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367711 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 12:44:18.372602 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367714 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367716 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367719 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367722 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367724 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367727 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367729 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367732 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367734 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367737 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367739 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367742 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367744 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367747 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367749 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367752 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367755 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367757 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367760 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367763 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 12:44:18.373075 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367766 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367769 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367771 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367773 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367776 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367778 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367781 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367783 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367786 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367788 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367791 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367794 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367796 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367799 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367801 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367804 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367807 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367810 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367812 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367814 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 12:44:18.373571 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:18.367817 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 12:44:18.374066 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.367823 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 12:44:18.374066 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.367962 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 12:44:18.374066 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.371626 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 12:44:18.374066 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.373866 2570 server.go:1019] "Starting client certificate rotation" Jul 20 12:44:18.374066 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.373973 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 12:44:18.374745 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.374732 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 12:44:18.376667 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.376651 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 12:44:18.379753 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.379736 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 12:44:18.393440 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.393235 2570 log.go:25] "Validated CRI v1 runtime API" Jul 20 12:44:18.396265 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.396248 2570 log.go:25] "Validated CRI v1 image API" Jul 20 12:44:18.397071 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.397054 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 12:44:18.400875 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.400853 2570 fs.go:135] Filesystem UUIDs: map[506c21a3-da7b-41a6-99e6-545301b0e3eb:/dev/nvme0n1p4 79f5d025-300b-48ff-bc2b-28d2ce673bde:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 20 12:44:18.400930 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.400874 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 12:44:18.404187 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.404166 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 12:44:18.406416 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.406277 2570 manager.go:216] Machine: {Timestamp:2026-07-20 12:44:18.40480112 +0000 UTC m=+0.377006942 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3043176 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2665ce4083287d4fd4a8ee7169a468 SystemUUID:ec2665ce-4083-287d-4fd4-a8ee7169a468 BootID:b7b9e778-232d-4768-ba9d-9d492552ab20 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:84:79:23:95:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:84:79:23:95:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:62:d1:f9:28:82 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 12:44:18.406416 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.406415 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 12:44:18.406575 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.406562 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 12:44:18.410422 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.410384 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 12:44:18.410583 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.410428 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-229.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 12:44:18.411275 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.411263 2570 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 12:44:18.411309 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.411277 2570 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 12:44:18.411309 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.411284 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 12:44:18.411309 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.411300 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 12:44:18.413058 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.413043 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 12:44:18.416559 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.416541 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 20 12:44:18.416698 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.416689 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 12:44:18.416728 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.416722 2570 kubelet.go:486] "Attempting to sync node with API server" Jul 20 12:44:18.416755 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.416732 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 12:44:18.416755 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.416747 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 12:44:18.416818 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.416757 2570 kubelet.go:398] "Adding apiserver pod source" Jul 20 12:44:18.416818 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.416767 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 12:44:18.417233 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.417214 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 12:44:18.417298 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.417239 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 12:44:18.420404 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.420363 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 12:44:18.421326 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.421307 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwlbv" Jul 20 12:44:18.422118 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.422104 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 12:44:18.422160 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.422135 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 12:44:18.422939 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.422926 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 12:44:18.422986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.422946 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 12:44:18.422986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.422954 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 12:44:18.422986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.422961 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 12:44:18.422986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.422973 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 12:44:18.422986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.422979 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 12:44:18.423184 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.423176 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 12:44:18.423184 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.423184 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 12:44:18.423242 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.423191 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 12:44:18.423242 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.423197 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 12:44:18.423242 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.423205 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 12:44:18.423242 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.423212 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 12:44:18.424886 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.424873 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 12:44:18.424919 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.424889 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 12:44:18.426773 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.426750 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 12:44:18.426842 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.426752 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 12:44:18.427531 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.427516 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-229.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 12:44:18.428430 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.428414 2570 server.go:1268] "Started kubelet" Jul 20 12:44:18.428550 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.428518 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 12:44:18.428606 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.428585 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 12:44:18.428651 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.428524 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 12:44:18.428890 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.428861 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwlbv" Jul 20 12:44:18.429480 ip-10-0-141-229 systemd[1]: Started Kubernetes Kubelet. Jul 20 12:44:18.430352 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.430335 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 12:44:18.430877 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.430861 2570 server.go:310] "Adding debug handlers to kubelet server" Jul 20 12:44:18.436613 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.436591 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 12:44:18.439215 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.437762 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-229.ec2.internal.18c40018b30772f9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-229.ec2.internal,UID:ip-10-0-141-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-229.ec2.internal,},FirstTimestamp:2026-07-20 12:44:18.428383993 +0000 UTC m=+0.400589815,LastTimestamp:2026-07-20 12:44:18.428383993 +0000 UTC m=+0.400589815,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-229.ec2.internal,}" Jul 20 12:44:18.442536 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.442521 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 12:44:18.442614 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.442535 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 12:44:18.442677 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.442660 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:18.443296 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443280 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 12:44:18.443432 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443416 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 12:44:18.443517 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443502 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 12:44:18.443631 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443434 2570 factory.go:55] Registering systemd factory Jul 20 12:44:18.443631 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443578 2570 factory.go:223] Registration of the systemd container factory successfully Jul 20 12:44:18.443733 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443639 2570 reconstruct.go:97] "Volume reconstruction finished" Jul 20 12:44:18.443733 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443648 2570 reconciler.go:29] "Reconciler: start to sync state" Jul 20 12:44:18.443812 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443804 2570 factory.go:153] Registering CRI-O factory Jul 20 12:44:18.443853 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443815 2570 factory.go:223] Registration of the crio container factory successfully Jul 20 12:44:18.443894 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443866 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 12:44:18.443894 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443884 2570 factory.go:103] Registering Raw factory Jul 20 12:44:18.443894 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.443894 2570 manager.go:1196] Started watching for new ooms in manager Jul 20 12:44:18.444306 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.444294 2570 manager.go:349] Starting recovery of all containers Jul 20 12:44:18.452759 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.452734 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 12:44:18.454919 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.454902 2570 manager.go:354] Recovery completed Jul 20 12:44:18.455561 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.455539 2570 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-229.ec2.internal\" not found" node="ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.458956 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.458943 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 12:44:18.461738 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.461714 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory" Jul 20 12:44:18.461869 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.461751 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 12:44:18.461869 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.461796 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID" Jul 20 12:44:18.462710 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.462691 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 12:44:18.462710 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.462707 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 12:44:18.462832 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.462714 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 12:44:18.462832 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.462729 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 20 12:44:18.463571 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.463555 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory" Jul 20 12:44:18.463657 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.463588 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 12:44:18.463657 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.463602 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID" Jul 20 12:44:18.463853 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.463841 2570 policy_none.go:49] "None policy: Start" Jul 20 12:44:18.463901 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.463856 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 12:44:18.463901 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.463866 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 12:44:18.464855 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.464844 2570 policy_none.go:47] "Start" Jul 20 12:44:18.512302 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.512037 2570 manager.go:337] "Starting Device Plugin manager" Jul 20 12:44:18.512302 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.512284 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 12:44:18.512302 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.512295 2570 server.go:87] "Starting device plugin registration server" Jul 20 12:44:18.514134 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.512554 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 12:44:18.514134 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.512565 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 12:44:18.514134 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.512734 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 12:44:18.514134 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.512815 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 12:44:18.514134 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.512823 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 12:44:18.514134 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.513093 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 12:44:18.514134 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.513355 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 12:44:18.514134 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.513416 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:18.514799 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.514772 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 12:44:18.514874 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.514812 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 12:44:18.514874 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.514840 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 12:44:18.514968 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.514881 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 12:44:18.517582 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.517561 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 12:44:18.613659 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.613548 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 12:44:18.614540 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.614523 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory" Jul 20 12:44:18.614631 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.614558 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 12:44:18.614631 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.614572 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID" Jul 20 12:44:18.614631 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.614597 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.615622 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.615593 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal"] Jul 20 12:44:18.615672 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.615664 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 12:44:18.616444 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.616419 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory" Jul 20 12:44:18.616503 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.616447 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 12:44:18.616503 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.616458 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID" Jul 20 12:44:18.617711 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.617698 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 12:44:18.617880 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.617863 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.617948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.617902 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 12:44:18.618335 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.618320 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory" Jul 20 12:44:18.618433 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.618346 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 12:44:18.618433 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.618357 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID" Jul 20 12:44:18.618433 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.618324 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory" Jul 20 12:44:18.618433 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.618400 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 12:44:18.618433 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.618411 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID" Jul 20 12:44:18.619449 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.619435 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.619498 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.619459 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 12:44:18.620071 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.620056 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory" Jul 20 12:44:18.620143 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.620083 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 12:44:18.620143 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.620093 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID" Jul 20 12:44:18.625682 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.625667 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.625741 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.625696 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-229.ec2.internal\": node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:18.640091 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.640065 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-229.ec2.internal\" not found" node="ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.643757 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.643738 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-229.ec2.internal\" not found" node="ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.644739 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.644721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92a19fc888e8076d0b3fbd4b9dc96a0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"92a19fc888e8076d0b3fbd4b9dc96a0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.644814 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.644753 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92a19fc888e8076d0b3fbd4b9dc96a0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"92a19fc888e8076d0b3fbd4b9dc96a0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.644814 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.644782 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/192fe6abc0214ce0c06c38f944481f45-config\") pod \"kube-apiserver-proxy-ip-10-0-141-229.ec2.internal\" (UID: \"192fe6abc0214ce0c06c38f944481f45\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.651512 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.651489 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:18.745055 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.745018 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/192fe6abc0214ce0c06c38f944481f45-config\") pod \"kube-apiserver-proxy-ip-10-0-141-229.ec2.internal\" (UID: \"192fe6abc0214ce0c06c38f944481f45\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.745055 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.745053 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92a19fc888e8076d0b3fbd4b9dc96a0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"92a19fc888e8076d0b3fbd4b9dc96a0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.745233 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.745071 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92a19fc888e8076d0b3fbd4b9dc96a0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"92a19fc888e8076d0b3fbd4b9dc96a0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.745233 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.745118 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/92a19fc888e8076d0b3fbd4b9dc96a0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"92a19fc888e8076d0b3fbd4b9dc96a0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.745233 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.745122 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/192fe6abc0214ce0c06c38f944481f45-config\") pod \"kube-apiserver-proxy-ip-10-0-141-229.ec2.internal\" (UID: \"192fe6abc0214ce0c06c38f944481f45\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.745233 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.745175 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92a19fc888e8076d0b3fbd4b9dc96a0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"92a19fc888e8076d0b3fbd4b9dc96a0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.752384 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.752345 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:18.853513 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.853477 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:18.942015 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.941956 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.949196 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:18.949176 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" Jul 20 12:44:18.954420 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:18.954400 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:19.054950 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:19.054903 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:19.155517 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:19.155488 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:19.256079 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:19.255997 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:19.356600 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:19.356572 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:19.372767 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.372738 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 12:44:19.372941 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.372919 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 12:44:19.372993 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.372948 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 12:44:19.438320 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.438271 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 12:39:19 +0000 UTC" deadline="2028-02-18 03:10:03.75205542 +0000 UTC" Jul 20 12:44:19.438320 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.438312 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13862h25m44.313747306s" Jul 20 12:44:19.445317 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:19.445287 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192fe6abc0214ce0c06c38f944481f45.slice/crio-617201e1fb120f899b2bed3a63c9b720cf1ff0bd5f6adc5f34759d91bbbb9f1f WatchSource:0}: Error finding container 617201e1fb120f899b2bed3a63c9b720cf1ff0bd5f6adc5f34759d91bbbb9f1f: Status 404 returned error can't find the container with id 617201e1fb120f899b2bed3a63c9b720cf1ff0bd5f6adc5f34759d91bbbb9f1f Jul 20 12:44:19.445654 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:19.445633 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a19fc888e8076d0b3fbd4b9dc96a0a.slice/crio-ea696bf15238c4b56ad8eb21445f15d39933741b08f99ba20031c818aa7a2696 WatchSource:0}: Error finding container ea696bf15238c4b56ad8eb21445f15d39933741b08f99ba20031c818aa7a2696: Status 404 returned error can't find the container with id ea696bf15238c4b56ad8eb21445f15d39933741b08f99ba20031c818aa7a2696 Jul 20 12:44:19.451847 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.451827 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 12:44:19.457290 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:19.457269 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:19.517898 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.517788 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" event={"ID":"92a19fc888e8076d0b3fbd4b9dc96a0a","Type":"ContainerStarted","Data":"ea696bf15238c4b56ad8eb21445f15d39933741b08f99ba20031c818aa7a2696"} Jul 20 12:44:19.518815 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.518783 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" event={"ID":"192fe6abc0214ce0c06c38f944481f45","Type":"ContainerStarted","Data":"617201e1fb120f899b2bed3a63c9b720cf1ff0bd5f6adc5f34759d91bbbb9f1f"} Jul 20 12:44:19.557383 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:19.557329 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:19.657890 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:19.657840 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:19.758438 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:19.758412 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:19.859022 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:19.858932 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found" Jul 20 12:44:19.861051 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.861022 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 12:44:19.942984 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.942949 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" Jul 20 12:44:19.953356 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.953324 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 12:44:19.954475 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.954454 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" Jul 20 12:44:19.955727 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.955700 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 12:44:19.963290 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:19.963266 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 12:44:20.417415 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.417348 2570 apiserver.go:52] "Watching apiserver" Jul 20 12:44:20.423125 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.423096 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 12:44:20.424667 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.424628 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-kpsm8","openshift-multus/network-metrics-daemon-6gkrz","openshift-network-diagnostics/network-check-target-67vbc","kube-system/konnectivity-agent-55hwm","kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal","openshift-network-operator/iptables-alerter-hx52w","openshift-ovn-kubernetes/ovnkube-node-z95dr","kube-system/global-pull-secret-syncer-wvc67","openshift-cluster-node-tuning-operator/tuned-8k96c","openshift-dns/node-resolver-jstg9","openshift-image-registry/node-ca-vs9qv","openshift-multus/multus-additional-cni-plugins-lhm8h"] Jul 20 12:44:20.426281 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.426253 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:20.426440 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:20.426340 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:20.428316 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.428202 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.429306 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.429283 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:20.429426 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:20.429362 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:20.430027 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.430009 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 12:44:20.430124 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.430011 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8w2p7\"" Jul 20 12:44:20.430511 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.430013 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 12:44:20.430787 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.430768 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:20.432740 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.432539 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lr8x6\"" Jul 20 12:44:20.433391 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.433012 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 12:44:20.433391 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.433209 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 12:44:20.433702 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.433607 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.435481 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.435460 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.435602 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.435547 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 12:44:20.436088 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.435853 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 12:44:20.436088 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.435867 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-llctl\"" Jul 20 12:44:20.436088 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.435867 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 12:44:20.436088 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.435932 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 12:44:20.436088 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.435932 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 12:44:20.437076 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.436993 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.437872 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.437853 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 12:44:20.438012 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.437996 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 12:44:20.438378 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.438345 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 12:44:20.438598 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.438581 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-96p8h\"" Jul 20 12:44:20.438836 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.438800 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vs9qv" Jul 20 12:44:20.438934 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.438912 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 12:44:20.439596 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.439558 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 12:44:20.439679 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.439663 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 12:44:20.439842 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.439826 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 12:44:20.439904 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.439874 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 12:44:20.440060 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.440044 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:20.440118 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.439561 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v6q6m\"" Jul 20 12:44:20.440168 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:20.440120 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:20.440409 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.440333 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 12:44:20.441196 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.440657 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-779f5\"" Jul 20 12:44:20.441196 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.440698 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 12:44:20.441196 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.440712 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 12:44:20.441196 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.440736 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 12:44:20.441196 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.441114 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.442644 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.442625 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hx52w" Jul 20 12:44:20.442738 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.442661 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 12:44:20.443388 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.443281 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4pdpw\"" Jul 20 12:44:20.443484 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.443466 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 12:44:20.443991 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.443973 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jstg9" Jul 20 12:44:20.444605 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.444464 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vtxbv\"" Jul 20 12:44:20.444605 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.444514 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 12:44:20.444741 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.444712 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 12:44:20.444790 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.444774 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 12:44:20.445564 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.445480 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 12:44:20.446210 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.446191 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 12:44:20.446292 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.446191 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pt7q6\"" Jul 20 12:44:20.451558 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.451528 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 12:44:20.453738 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.453721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-registration-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.453824 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.453754 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-kubelet\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.453824 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.453780 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-node-log\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.453928 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.453836 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd720144-9924-46ef-a880-0c196604451b-host\") pod \"node-ca-vs9qv\" (UID: \"fd720144-9924-46ef-a880-0c196604451b\") " pod="openshift-image-registry/node-ca-vs9qv" Jul 20 12:44:20.453928 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.453873 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-sys-fs\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.454028 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.453921 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-run-netns\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.454028 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.453958 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9eb5f9c7-9a60-4075-b022-e5308b67d79c-ovn-node-metrics-cert\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.454028 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.453987 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-cni-dir\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.454028 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454011 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/717fdc62-4efb-490e-bd34-e147ac34ddbf-cni-binary-copy\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.454210 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454040 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-hostroot\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.454210 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454069 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-tuned\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.454210 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454109 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rxcw\" (UniqueName: \"kubernetes.io/projected/1910d8dc-faf0-408f-8639-b05dc8bb0769-kube-api-access-9rxcw\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.454210 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454144 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-systemd-units\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.454210 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454176 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-conf-dir\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.454446 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454215 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-sysconfig\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.454446 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454249 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-var-lib-kubelet\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.454446 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454295 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fc1cf019-6b91-49b4-b73a-1c41a257269d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.454446 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454332 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-run-ovn\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.454446 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454356 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9eb5f9c7-9a60-4075-b022-e5308b67d79c-ovnkube-config\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.454446 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454395 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-run-multus-certs\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.454446 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454419 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g6pt\" (UniqueName: \"kubernetes.io/projected/29168ce6-87e8-41ea-aa38-d9e184ffb90b-kube-api-access-6g6pt\") pod \"iptables-alerter-hx52w\" (UID: \"29168ce6-87e8-41ea-aa38-d9e184ffb90b\") " pod="openshift-network-operator/iptables-alerter-hx52w" Jul 20 12:44:20.454446 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454441 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-systemd\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.454756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454462 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9470643-a690-4466-b8ec-496ccf18c5e6-tmp\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.454756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454486 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:20.454756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454507 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1fc1557-eab5-4747-9378-59a27b43a32a-tmp-dir\") pod \"node-resolver-jstg9\" (UID: \"c1fc1557-eab5-4747-9378-59a27b43a32a\") " pod="openshift-dns/node-resolver-jstg9" Jul 20 12:44:20.454756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454542 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-cnibin\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.454756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454585 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-slash\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.454756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454615 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlnh4\" (UniqueName: \"kubernetes.io/projected/9eb5f9c7-9a60-4075-b022-e5308b67d79c-kube-api-access-xlnh4\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.454756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454635 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-etc-kubernetes\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.454756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454652 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/29168ce6-87e8-41ea-aa38-d9e184ffb90b-iptables-alerter-script\") pod \"iptables-alerter-hx52w\" (UID: \"29168ce6-87e8-41ea-aa38-d9e184ffb90b\") " pod="openshift-network-operator/iptables-alerter-hx52w" Jul 20 12:44:20.454756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454674 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9eb5f9c7-9a60-4075-b022-e5308b67d79c-ovnkube-script-lib\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.454756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454698 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtxxk\" (UniqueName: \"kubernetes.io/projected/fd720144-9924-46ef-a880-0c196604451b-kube-api-access-xtxxk\") pod \"node-ca-vs9qv\" (UID: \"fd720144-9924-46ef-a880-0c196604451b\") " pod="openshift-image-registry/node-ca-vs9qv" Jul 20 12:44:20.454756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454742 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1220141b-d9f7-4e87-95e8-cc658b486b7f-dbus\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454778 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454803 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-modprobe-d\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454824 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-run\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454847 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-os-release\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454869 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454910 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-etc-selinux\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454944 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-var-lib-openvswitch\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.454968 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-run-k8s-cni-cncf-io\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455028 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lmgc\" (UniqueName: \"kubernetes.io/projected/24ad9090-71b2-4f16-9389-19f115738cf6-kube-api-access-9lmgc\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455055 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-run-openvswitch\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455080 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-run-systemd\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455102 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-log-socket\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455126 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-daemon-config\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455148 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-run-ovn-kubernetes\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455187 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29168ce6-87e8-41ea-aa38-d9e184ffb90b-host-slash\") pod \"iptables-alerter-hx52w\" (UID: \"29168ce6-87e8-41ea-aa38-d9e184ffb90b\") " pod="openshift-network-operator/iptables-alerter-hx52w" Jul 20 12:44:20.455240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455220 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455246 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-system-cni-dir\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455275 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-var-lib-cni-multus\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455294 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-var-lib-kubelet\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455309 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1220141b-d9f7-4e87-95e8-cc658b486b7f-kubelet-config\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455323 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-sysctl-conf\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455341 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl2w5\" (UniqueName: \"kubernetes.io/projected/c9470643-a690-4466-b8ec-496ccf18c5e6-kube-api-access-bl2w5\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455381 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fc1cf019-6b91-49b4-b73a-1c41a257269d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455405 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-cni-bin\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455433 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9eb5f9c7-9a60-4075-b022-e5308b67d79c-env-overrides\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455454 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-sysctl-d\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455469 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-host\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455495 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/36c5fc1a-f0e9-4443-b922-cd84c06112e5-konnectivity-ca\") pod \"konnectivity-agent-55hwm\" (UID: \"36c5fc1a-f0e9-4443-b922-cd84c06112e5\") " pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455510 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-system-cni-dir\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455528 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-os-release\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455544 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-run-netns\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.455915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455564 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-var-lib-cni-bin\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455587 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f5dz\" (UniqueName: \"kubernetes.io/projected/717fdc62-4efb-490e-bd34-e147ac34ddbf-kube-api-access-9f5dz\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455612 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-device-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455642 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-cni-netd\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455663 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455683 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-cnibin\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455699 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5gvb\" (UniqueName: \"kubernetes.io/projected/c1fc1557-eab5-4747-9378-59a27b43a32a-kube-api-access-h5gvb\") pod \"node-resolver-jstg9\" (UID: \"c1fc1557-eab5-4747-9378-59a27b43a32a\") " pod="openshift-dns/node-resolver-jstg9" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455714 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-socket-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455730 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fd720144-9924-46ef-a880-0c196604451b-serviceca\") pod \"node-ca-vs9qv\" (UID: \"fd720144-9924-46ef-a880-0c196604451b\") " pod="openshift-image-registry/node-ca-vs9qv" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455765 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-socket-dir-parent\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455794 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-kubernetes\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455819 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-lib-modules\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455842 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1fc1557-eab5-4747-9378-59a27b43a32a-hosts-file\") pod \"node-resolver-jstg9\" (UID: \"c1fc1557-eab5-4747-9378-59a27b43a32a\") " pod="openshift-dns/node-resolver-jstg9" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455859 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-sys\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455878 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbvgj\" (UniqueName: \"kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj\") pod \"network-check-target-67vbc\" (UID: \"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72\") " pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455920 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-etc-openvswitch\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.456421 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455960 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/36c5fc1a-f0e9-4443-b922-cd84c06112e5-agent-certs\") pod \"konnectivity-agent-55hwm\" (UID: \"36c5fc1a-f0e9-4443-b922-cd84c06112e5\") " pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:20.457025 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.455986 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc1cf019-6b91-49b4-b73a-1c41a257269d-cni-binary-copy\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.457025 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.456010 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nxk8\" (UniqueName: \"kubernetes.io/projected/fc1cf019-6b91-49b4-b73a-1c41a257269d-kube-api-access-6nxk8\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.470690 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.470657 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6cr5q" Jul 20 12:44:20.480405 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.480340 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6cr5q" Jul 20 12:44:20.544641 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.544600 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 12:44:20.556414 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556377 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-modprobe-d\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.556586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556423 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-run\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.556586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556451 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-os-release\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.556586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556479 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.556586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556502 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-etc-selinux\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.556586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556526 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-var-lib-openvswitch\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.556586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556540 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-modprobe-d\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.556586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556556 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-run\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.556586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556549 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-run-k8s-cni-cncf-io\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.556586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556585 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556593 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-os-release\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556604 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-run-k8s-cni-cncf-io\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556604 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lmgc\" (UniqueName: \"kubernetes.io/projected/24ad9090-71b2-4f16-9389-19f115738cf6-kube-api-access-9lmgc\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556638 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-var-lib-openvswitch\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556671 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-run-openvswitch\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556676 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-etc-selinux\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556710 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-run-systemd\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556750 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-run-systemd\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556787 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-run-openvswitch\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556815 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-log-socket\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556845 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-daemon-config\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556862 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-run-ovn-kubernetes\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556878 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29168ce6-87e8-41ea-aa38-d9e184ffb90b-host-slash\") pod \"iptables-alerter-hx52w\" (UID: \"29168ce6-87e8-41ea-aa38-d9e184ffb90b\") " pod="openshift-network-operator/iptables-alerter-hx52w" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556911 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556933 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-log-socket\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.556948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556937 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-run-ovn-kubernetes\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556957 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29168ce6-87e8-41ea-aa38-d9e184ffb90b-host-slash\") pod \"iptables-alerter-hx52w\" (UID: \"29168ce6-87e8-41ea-aa38-d9e184ffb90b\") " pod="openshift-network-operator/iptables-alerter-hx52w" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.556996 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-system-cni-dir\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557022 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-var-lib-cni-multus\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557049 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-var-lib-kubelet\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557059 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-system-cni-dir\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557074 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1220141b-d9f7-4e87-95e8-cc658b486b7f-kubelet-config\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557099 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-sysctl-conf\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557102 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-var-lib-cni-multus\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557105 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557122 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl2w5\" (UniqueName: \"kubernetes.io/projected/c9470643-a690-4466-b8ec-496ccf18c5e6-kube-api-access-bl2w5\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557143 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-var-lib-kubelet\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557151 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1220141b-d9f7-4e87-95e8-cc658b486b7f-kubelet-config\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557150 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fc1cf019-6b91-49b4-b73a-1c41a257269d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557190 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-cni-bin\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557213 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9eb5f9c7-9a60-4075-b022-e5308b67d79c-env-overrides\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557229 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-sysctl-conf\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.557730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557237 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-sysctl-d\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557264 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-host\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557284 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/36c5fc1a-f0e9-4443-b922-cd84c06112e5-konnectivity-ca\") pod \"konnectivity-agent-55hwm\" (UID: \"36c5fc1a-f0e9-4443-b922-cd84c06112e5\") " pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557304 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-system-cni-dir\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557319 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-os-release\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557329 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-sysctl-d\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557342 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-run-netns\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557357 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-var-lib-cni-bin\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557395 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f5dz\" (UniqueName: \"kubernetes.io/projected/717fdc62-4efb-490e-bd34-e147ac34ddbf-kube-api-access-9f5dz\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557403 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-host\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557412 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-device-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557441 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-cni-netd\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557473 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-device-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557481 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-daemon-config\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557494 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557524 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-cnibin\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557532 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-run-netns\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557444 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-var-lib-cni-bin\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.558515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557573 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-cnibin\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557575 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5gvb\" (UniqueName: \"kubernetes.io/projected/c1fc1557-eab5-4747-9378-59a27b43a32a-kube-api-access-h5gvb\") pod \"node-resolver-jstg9\" (UID: \"c1fc1557-eab5-4747-9378-59a27b43a32a\") " pod="openshift-dns/node-resolver-jstg9" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557590 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-system-cni-dir\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557601 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-socket-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557606 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557669 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-os-release\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557686 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-socket-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557709 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fc1cf019-6b91-49b4-b73a-1c41a257269d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557752 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fd720144-9924-46ef-a880-0c196604451b-serviceca\") pod \"node-ca-vs9qv\" (UID: \"fd720144-9924-46ef-a880-0c196604451b\") " pod="openshift-image-registry/node-ca-vs9qv" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557781 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-socket-dir-parent\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557795 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9eb5f9c7-9a60-4075-b022-e5308b67d79c-env-overrides\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557796 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-cni-netd\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557816 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-kubernetes\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557855 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-lib-modules\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557870 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-kubernetes\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557879 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1fc1557-eab5-4747-9378-59a27b43a32a-hosts-file\") pod \"node-resolver-jstg9\" (UID: \"c1fc1557-eab5-4747-9378-59a27b43a32a\") " pod="openshift-dns/node-resolver-jstg9" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557889 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-cni-bin\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.559393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557889 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-socket-dir-parent\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557901 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-sys\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557933 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1fc1557-eab5-4747-9378-59a27b43a32a-hosts-file\") pod \"node-resolver-jstg9\" (UID: \"c1fc1557-eab5-4747-9378-59a27b43a32a\") " pod="openshift-dns/node-resolver-jstg9" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557940 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-sys\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557952 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbvgj\" (UniqueName: \"kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj\") pod \"network-check-target-67vbc\" (UID: \"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72\") " pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557956 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-lib-modules\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.557997 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-etc-openvswitch\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558016 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/36c5fc1a-f0e9-4443-b922-cd84c06112e5-agent-certs\") pod \"konnectivity-agent-55hwm\" (UID: \"36c5fc1a-f0e9-4443-b922-cd84c06112e5\") " pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558023 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/36c5fc1a-f0e9-4443-b922-cd84c06112e5-konnectivity-ca\") pod \"konnectivity-agent-55hwm\" (UID: \"36c5fc1a-f0e9-4443-b922-cd84c06112e5\") " pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558040 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc1cf019-6b91-49b4-b73a-1c41a257269d-cni-binary-copy\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558081 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nxk8\" (UniqueName: \"kubernetes.io/projected/fc1cf019-6b91-49b4-b73a-1c41a257269d-kube-api-access-6nxk8\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558105 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-registration-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558081 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-etc-openvswitch\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558130 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-kubelet\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558189 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-node-log\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558198 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-registration-dir\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558226 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd720144-9924-46ef-a880-0c196604451b-host\") pod \"node-ca-vs9qv\" (UID: \"fd720144-9924-46ef-a880-0c196604451b\") " pod="openshift-image-registry/node-ca-vs9qv" Jul 20 12:44:20.560229 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558243 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-node-log\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558247 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-kubelet\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558292 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd720144-9924-46ef-a880-0c196604451b-host\") pod \"node-ca-vs9qv\" (UID: \"fd720144-9924-46ef-a880-0c196604451b\") " pod="openshift-image-registry/node-ca-vs9qv" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558308 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-sys-fs\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558232 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fd720144-9924-46ef-a880-0c196604451b-serviceca\") pod \"node-ca-vs9qv\" (UID: \"fd720144-9924-46ef-a880-0c196604451b\") " pod="openshift-image-registry/node-ca-vs9qv" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558254 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1910d8dc-faf0-408f-8639-b05dc8bb0769-sys-fs\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558360 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-run-netns\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558363 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558406 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9eb5f9c7-9a60-4075-b022-e5308b67d79c-ovn-node-metrics-cert\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558415 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-run-netns\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558429 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-cni-dir\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558474 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/717fdc62-4efb-490e-bd34-e147ac34ddbf-cni-binary-copy\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558497 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-hostroot\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558518 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-tuned\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558543 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rxcw\" (UniqueName: \"kubernetes.io/projected/1910d8dc-faf0-408f-8639-b05dc8bb0769-kube-api-access-9rxcw\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558572 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-systemd-units\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558574 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc1cf019-6b91-49b4-b73a-1c41a257269d-cni-binary-copy\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558596 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-conf-dir\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.560986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558621 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-sysconfig\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558644 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-var-lib-kubelet\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558691 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fc1cf019-6b91-49b4-b73a-1c41a257269d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558718 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-run-ovn\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558744 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9eb5f9c7-9a60-4075-b022-e5308b67d79c-ovnkube-config\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558754 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-hostroot\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558767 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-run-multus-certs\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558794 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g6pt\" (UniqueName: \"kubernetes.io/projected/29168ce6-87e8-41ea-aa38-d9e184ffb90b-kube-api-access-6g6pt\") pod \"iptables-alerter-hx52w\" (UID: \"29168ce6-87e8-41ea-aa38-d9e184ffb90b\") " pod="openshift-network-operator/iptables-alerter-hx52w" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558817 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-systemd\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558840 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9470643-a690-4466-b8ec-496ccf18c5e6-tmp\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558860 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558869 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-cni-dir\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558882 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1fc1557-eab5-4747-9378-59a27b43a32a-tmp-dir\") pod \"node-resolver-jstg9\" (UID: \"c1fc1557-eab5-4747-9378-59a27b43a32a\") " pod="openshift-dns/node-resolver-jstg9" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558924 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-cnibin\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558926 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-run-ovn\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558965 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-slash\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.558993 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlnh4\" (UniqueName: \"kubernetes.io/projected/9eb5f9c7-9a60-4075-b022-e5308b67d79c-kube-api-access-xlnh4\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559014 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/717fdc62-4efb-490e-bd34-e147ac34ddbf-cni-binary-copy\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.561957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559026 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-etc-kubernetes\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559054 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/29168ce6-87e8-41ea-aa38-d9e184ffb90b-iptables-alerter-script\") pod \"iptables-alerter-hx52w\" (UID: \"29168ce6-87e8-41ea-aa38-d9e184ffb90b\") " pod="openshift-network-operator/iptables-alerter-hx52w" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559080 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9eb5f9c7-9a60-4075-b022-e5308b67d79c-ovnkube-script-lib\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559105 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtxxk\" (UniqueName: \"kubernetes.io/projected/fd720144-9924-46ef-a880-0c196604451b-kube-api-access-xtxxk\") pod \"node-ca-vs9qv\" (UID: \"fd720144-9924-46ef-a880-0c196604451b\") " pod="openshift-image-registry/node-ca-vs9qv" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559130 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1220141b-d9f7-4e87-95e8-cc658b486b7f-dbus\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559162 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-systemd-units\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559174 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559210 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-multus-conf-dir\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559262 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-sysconfig\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:20.559291 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559313 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-var-lib-kubelet\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:20.559393 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret podName:1220141b-d9f7-4e87-95e8-cc658b486b7f nodeName:}" failed. No retries permitted until 2026-07-20 12:44:21.059340206 +0000 UTC m=+3.031546019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret") pod "global-pull-secret-syncer-wvc67" (UID: "1220141b-d9f7-4e87-95e8-cc658b486b7f") : object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559463 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9eb5f9c7-9a60-4075-b022-e5308b67d79c-ovnkube-config\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559523 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-host-run-multus-certs\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559597 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-systemd\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559795 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fc1cf019-6b91-49b4-b73a-1c41a257269d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.559833 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1fc1557-eab5-4747-9378-59a27b43a32a-tmp-dir\") pod \"node-resolver-jstg9\" (UID: \"c1fc1557-eab5-4747-9378-59a27b43a32a\") " pod="openshift-dns/node-resolver-jstg9" Jul 20 12:44:20.562797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.560071 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1220141b-d9f7-4e87-95e8-cc658b486b7f-dbus\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:20.563731 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.560091 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/29168ce6-87e8-41ea-aa38-d9e184ffb90b-iptables-alerter-script\") pod \"iptables-alerter-hx52w\" (UID: \"29168ce6-87e8-41ea-aa38-d9e184ffb90b\") " pod="openshift-network-operator/iptables-alerter-hx52w" Jul 20 12:44:20.563731 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.560148 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc1cf019-6b91-49b4-b73a-1c41a257269d-cnibin\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.563731 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.560185 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9eb5f9c7-9a60-4075-b022-e5308b67d79c-host-slash\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.563731 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:20.560286 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:20.563731 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.560332 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9eb5f9c7-9a60-4075-b022-e5308b67d79c-ovnkube-script-lib\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.563731 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:20.560338 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs podName:24ad9090-71b2-4f16-9389-19f115738cf6 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:21.060321633 +0000 UTC m=+3.032527449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs") pod "network-metrics-daemon-6gkrz" (UID: "24ad9090-71b2-4f16-9389-19f115738cf6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:20.563731 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.560408 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/717fdc62-4efb-490e-bd34-e147ac34ddbf-etc-kubernetes\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.563731 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.562475 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9470643-a690-4466-b8ec-496ccf18c5e6-etc-tuned\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.563731 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.562561 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9eb5f9c7-9a60-4075-b022-e5308b67d79c-ovn-node-metrics-cert\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.563731 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.562736 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/36c5fc1a-f0e9-4443-b922-cd84c06112e5-agent-certs\") pod \"konnectivity-agent-55hwm\" (UID: \"36c5fc1a-f0e9-4443-b922-cd84c06112e5\") " pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:20.563731 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.563079 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9470643-a690-4466-b8ec-496ccf18c5e6-tmp\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.568802 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:20.568775 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 12:44:20.568802 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:20.568801 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 12:44:20.569080 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:20.568822 2570 projected.go:196] Error preparing data for projected volume kube-api-access-nbvgj for pod openshift-network-diagnostics/network-check-target-67vbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:20.569080 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:20.568882 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj podName:6e808ebf-fa3a-423a-a77f-5ae8d13c6e72 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:21.068863602 +0000 UTC m=+3.041069411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nbvgj" (UniqueName: "kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj") pod "network-check-target-67vbc" (UID: "6e808ebf-fa3a-423a-a77f-5ae8d13c6e72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:20.572440 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.572074 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5gvb\" (UniqueName: \"kubernetes.io/projected/c1fc1557-eab5-4747-9378-59a27b43a32a-kube-api-access-h5gvb\") pod \"node-resolver-jstg9\" (UID: \"c1fc1557-eab5-4747-9378-59a27b43a32a\") " pod="openshift-dns/node-resolver-jstg9" Jul 20 12:44:20.572440 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.572267 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl2w5\" (UniqueName: \"kubernetes.io/projected/c9470643-a690-4466-b8ec-496ccf18c5e6-kube-api-access-bl2w5\") pod \"tuned-8k96c\" (UID: \"c9470643-a690-4466-b8ec-496ccf18c5e6\") " pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.572440 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.572332 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rxcw\" (UniqueName: \"kubernetes.io/projected/1910d8dc-faf0-408f-8639-b05dc8bb0769-kube-api-access-9rxcw\") pod \"aws-ebs-csi-driver-node-t6zdn\" (UID: \"1910d8dc-faf0-408f-8639-b05dc8bb0769\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.572820 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.572750 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtxxk\" (UniqueName: \"kubernetes.io/projected/fd720144-9924-46ef-a880-0c196604451b-kube-api-access-xtxxk\") pod \"node-ca-vs9qv\" (UID: \"fd720144-9924-46ef-a880-0c196604451b\") " pod="openshift-image-registry/node-ca-vs9qv" Jul 20 12:44:20.573287 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.573269 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lmgc\" (UniqueName: \"kubernetes.io/projected/24ad9090-71b2-4f16-9389-19f115738cf6-kube-api-access-9lmgc\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:20.573850 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.573825 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g6pt\" (UniqueName: \"kubernetes.io/projected/29168ce6-87e8-41ea-aa38-d9e184ffb90b-kube-api-access-6g6pt\") pod \"iptables-alerter-hx52w\" (UID: \"29168ce6-87e8-41ea-aa38-d9e184ffb90b\") " pod="openshift-network-operator/iptables-alerter-hx52w" Jul 20 12:44:20.574546 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.574516 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nxk8\" (UniqueName: \"kubernetes.io/projected/fc1cf019-6b91-49b4-b73a-1c41a257269d-kube-api-access-6nxk8\") pod \"multus-additional-cni-plugins-lhm8h\" (UID: \"fc1cf019-6b91-49b4-b73a-1c41a257269d\") " pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.575001 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.574979 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f5dz\" (UniqueName: \"kubernetes.io/projected/717fdc62-4efb-490e-bd34-e147ac34ddbf-kube-api-access-9f5dz\") pod \"multus-kpsm8\" (UID: \"717fdc62-4efb-490e-bd34-e147ac34ddbf\") " pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.575217 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.575194 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlnh4\" (UniqueName: \"kubernetes.io/projected/9eb5f9c7-9a60-4075-b022-e5308b67d79c-kube-api-access-xlnh4\") pod \"ovnkube-node-z95dr\" (UID: \"9eb5f9c7-9a60-4075-b022-e5308b67d79c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.741843 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.741729 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8k96c" Jul 20 12:44:20.750485 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.750452 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:20.759183 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.759153 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lhm8h" Jul 20 12:44:20.766085 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.766054 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" Jul 20 12:44:20.776228 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.776202 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:20.783232 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.783210 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vs9qv" Jul 20 12:44:20.790557 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.790530 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kpsm8" Jul 20 12:44:20.799621 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.799593 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hx52w" Jul 20 12:44:20.805505 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.805483 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jstg9" Jul 20 12:44:20.807507 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.807481 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 12:44:20.933449 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:20.933404 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 12:44:21.063314 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.063218 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:21.063314 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.063271 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:21.063592 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:21.063388 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:21.063592 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:21.063403 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:21.063592 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:21.063482 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret podName:1220141b-d9f7-4e87-95e8-cc658b486b7f nodeName:}" failed. No retries permitted until 2026-07-20 12:44:22.063461107 +0000 UTC m=+4.035666918 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret") pod "global-pull-secret-syncer-wvc67" (UID: "1220141b-d9f7-4e87-95e8-cc658b486b7f") : object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:21.063592 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:21.063504 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs podName:24ad9090-71b2-4f16-9389-19f115738cf6 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:22.063494352 +0000 UTC m=+4.035700166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs") pod "network-metrics-daemon-6gkrz" (UID: "24ad9090-71b2-4f16-9389-19f115738cf6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:21.164316 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.164277 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbvgj\" (UniqueName: \"kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj\") pod \"network-check-target-67vbc\" (UID: \"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72\") " pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:21.164528 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:21.164443 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 12:44:21.164528 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:21.164468 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 12:44:21.164528 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:21.164481 2570 projected.go:196] Error preparing data for projected volume kube-api-access-nbvgj for pod openshift-network-diagnostics/network-check-target-67vbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:21.164729 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:21.164539 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj podName:6e808ebf-fa3a-423a-a77f-5ae8d13c6e72 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:22.164523487 +0000 UTC m=+4.136729299 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nbvgj" (UniqueName: "kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj") pod "network-check-target-67vbc" (UID: "6e808ebf-fa3a-423a-a77f-5ae8d13c6e72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:21.347464 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:21.347434 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb5f9c7_9a60_4075_b022_e5308b67d79c.slice/crio-ff300aabecfa25b927db7bcff91152961df083949f94d994bb122a5c2e852c03 WatchSource:0}: Error finding container ff300aabecfa25b927db7bcff91152961df083949f94d994bb122a5c2e852c03: Status 404 returned error can't find the container with id ff300aabecfa25b927db7bcff91152961df083949f94d994bb122a5c2e852c03 Jul 20 12:44:21.348500 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:21.348399 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd720144_9924_46ef_a880_0c196604451b.slice/crio-df41e359eb30370ef7db429a5b508b4c6c583c66826f3b42a8c51bd64d4d2a39 WatchSource:0}: Error finding container df41e359eb30370ef7db429a5b508b4c6c583c66826f3b42a8c51bd64d4d2a39: Status 404 returned error can't find the container with id df41e359eb30370ef7db429a5b508b4c6c583c66826f3b42a8c51bd64d4d2a39 Jul 20 12:44:21.349918 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:21.349891 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc1cf019_6b91_49b4_b73a_1c41a257269d.slice/crio-fb43b615cffca67cdc78295fcd8f02b56ebcd2914f5e6602dd13d6547572ebaa WatchSource:0}: Error finding container fb43b615cffca67cdc78295fcd8f02b56ebcd2914f5e6602dd13d6547572ebaa: Status 404 returned error can't find the container with id fb43b615cffca67cdc78295fcd8f02b56ebcd2914f5e6602dd13d6547572ebaa Jul 20 12:44:21.355304 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:21.355279 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1910d8dc_faf0_408f_8639_b05dc8bb0769.slice/crio-85f2d042256992cfdf73a4b933c283c6f84c5502f0274c6018f6f52d52137810 WatchSource:0}: Error finding container 85f2d042256992cfdf73a4b933c283c6f84c5502f0274c6018f6f52d52137810: Status 404 returned error can't find the container with id 85f2d042256992cfdf73a4b933c283c6f84c5502f0274c6018f6f52d52137810 Jul 20 12:44:21.356018 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:21.355981 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29168ce6_87e8_41ea_aa38_d9e184ffb90b.slice/crio-1a634d27502bd2a8eee1e51bc5039e1d21b18e48cb19f7fc20e7b6ed58dfe4c8 WatchSource:0}: Error finding container 1a634d27502bd2a8eee1e51bc5039e1d21b18e48cb19f7fc20e7b6ed58dfe4c8: Status 404 returned error can't find the container with id 1a634d27502bd2a8eee1e51bc5039e1d21b18e48cb19f7fc20e7b6ed58dfe4c8 Jul 20 12:44:21.356894 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:21.356866 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36c5fc1a_f0e9_4443_b922_cd84c06112e5.slice/crio-1f9b5dcd50e090e725802b6653c18198463ae26789ef1d95827f694adbfd8677 WatchSource:0}: Error finding container 1f9b5dcd50e090e725802b6653c18198463ae26789ef1d95827f694adbfd8677: Status 404 returned error can't find the container with id 1f9b5dcd50e090e725802b6653c18198463ae26789ef1d95827f694adbfd8677 Jul 20 12:44:21.359960 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:21.358329 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9470643_a690_4466_b8ec_496ccf18c5e6.slice/crio-69e743afd7cbd7b113de7ba23ce064487dbc1dc81443db89839cc69419883d88 WatchSource:0}: Error finding container 69e743afd7cbd7b113de7ba23ce064487dbc1dc81443db89839cc69419883d88: Status 404 returned error can't find the container with id 69e743afd7cbd7b113de7ba23ce064487dbc1dc81443db89839cc69419883d88 Jul 20 12:44:21.359960 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:21.359188 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1fc1557_eab5_4747_9378_59a27b43a32a.slice/crio-7bc6d22a270bda7910da69d5901006b1555abbb522d37a5dfc60d277152b6450 WatchSource:0}: Error finding container 7bc6d22a270bda7910da69d5901006b1555abbb522d37a5dfc60d277152b6450: Status 404 returned error can't find the container with id 7bc6d22a270bda7910da69d5901006b1555abbb522d37a5dfc60d277152b6450 Jul 20 12:44:21.484610 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.484549 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 12:39:21 +0000 UTC" deadline="2027-12-17 00:29:39.153712918 +0000 UTC" Jul 20 12:44:21.484610 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.484606 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12347h45m17.669111685s" Jul 20 12:44:21.515590 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.515559 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:21.515756 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:21.515677 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:21.526496 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.526459 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" event={"ID":"192fe6abc0214ce0c06c38f944481f45","Type":"ContainerStarted","Data":"6eea2ef8c88107a5e0e73a10e5a7ff3144d049e52e8087bec49263c307a236d9"} Jul 20 12:44:21.528594 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.528558 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8k96c" event={"ID":"c9470643-a690-4466-b8ec-496ccf18c5e6","Type":"ContainerStarted","Data":"69e743afd7cbd7b113de7ba23ce064487dbc1dc81443db89839cc69419883d88"} Jul 20 12:44:21.529850 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.529826 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" event={"ID":"1910d8dc-faf0-408f-8639-b05dc8bb0769","Type":"ContainerStarted","Data":"85f2d042256992cfdf73a4b933c283c6f84c5502f0274c6018f6f52d52137810"} Jul 20 12:44:21.531065 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.531028 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vs9qv" event={"ID":"fd720144-9924-46ef-a880-0c196604451b","Type":"ContainerStarted","Data":"df41e359eb30370ef7db429a5b508b4c6c583c66826f3b42a8c51bd64d4d2a39"} Jul 20 12:44:21.532210 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.532187 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kpsm8" event={"ID":"717fdc62-4efb-490e-bd34-e147ac34ddbf","Type":"ContainerStarted","Data":"652ba597d03cb1162c8a7650a40251a3715d93452c802feca773e5f5279a29c5"} Jul 20 12:44:21.533211 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.533178 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhm8h" event={"ID":"fc1cf019-6b91-49b4-b73a-1c41a257269d","Type":"ContainerStarted","Data":"fb43b615cffca67cdc78295fcd8f02b56ebcd2914f5e6602dd13d6547572ebaa"} Jul 20 12:44:21.534286 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.534265 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jstg9" event={"ID":"c1fc1557-eab5-4747-9378-59a27b43a32a","Type":"ContainerStarted","Data":"7bc6d22a270bda7910da69d5901006b1555abbb522d37a5dfc60d277152b6450"} Jul 20 12:44:21.535818 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.535795 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-55hwm" event={"ID":"36c5fc1a-f0e9-4443-b922-cd84c06112e5","Type":"ContainerStarted","Data":"1f9b5dcd50e090e725802b6653c18198463ae26789ef1d95827f694adbfd8677"} Jul 20 12:44:21.537788 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.537768 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hx52w" event={"ID":"29168ce6-87e8-41ea-aa38-d9e184ffb90b","Type":"ContainerStarted","Data":"1a634d27502bd2a8eee1e51bc5039e1d21b18e48cb19f7fc20e7b6ed58dfe4c8"} Jul 20 12:44:21.538854 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.538834 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" event={"ID":"9eb5f9c7-9a60-4075-b022-e5308b67d79c","Type":"ContainerStarted","Data":"ff300aabecfa25b927db7bcff91152961df083949f94d994bb122a5c2e852c03"} Jul 20 12:44:21.540311 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:21.540267 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" podStartSLOduration=1.5402539769999999 podStartE2EDuration="1.540253977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:44:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:21.540104899 +0000 UTC m=+3.512310735" watchObservedRunningTime="2026-07-20 12:44:21.540253977 +0000 UTC m=+3.512459812" Jul 20 12:44:22.072957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:22.072509 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:22.072957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:22.072829 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:22.073229 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:22.073006 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:22.073229 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:22.073075 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret podName:1220141b-d9f7-4e87-95e8-cc658b486b7f nodeName:}" failed. No retries permitted until 2026-07-20 12:44:24.073054933 +0000 UTC m=+6.045260749 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret") pod "global-pull-secret-syncer-wvc67" (UID: "1220141b-d9f7-4e87-95e8-cc658b486b7f") : object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:22.073534 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:22.073514 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:22.073624 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:22.073571 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs podName:24ad9090-71b2-4f16-9389-19f115738cf6 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:24.073555066 +0000 UTC m=+6.045760893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs") pod "network-metrics-daemon-6gkrz" (UID: "24ad9090-71b2-4f16-9389-19f115738cf6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:22.174394 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:22.173687 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbvgj\" (UniqueName: \"kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj\") pod \"network-check-target-67vbc\" (UID: \"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72\") " pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:22.174394 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:22.173871 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 12:44:22.174394 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:22.173898 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 12:44:22.174394 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:22.173913 2570 projected.go:196] Error preparing data for projected volume kube-api-access-nbvgj for pod openshift-network-diagnostics/network-check-target-67vbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:22.174394 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:22.173983 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj podName:6e808ebf-fa3a-423a-a77f-5ae8d13c6e72 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:24.173960472 +0000 UTC m=+6.146166286 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nbvgj" (UniqueName: "kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj") pod "network-check-target-67vbc" (UID: "6e808ebf-fa3a-423a-a77f-5ae8d13c6e72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:22.485391 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:22.485180 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 12:39:21 +0000 UTC" deadline="2028-04-06 18:40:49.07080544 +0000 UTC" Jul 20 12:44:22.485391 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:22.485248 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15029h56m26.58556175s" Jul 20 12:44:22.520420 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:22.519604 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:22.520420 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:22.519766 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:22.520420 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:22.520167 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:22.520420 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:22.520257 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:22.588983 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:22.588944 2570 generic.go:356] "Generic (PLEG): container finished" podID="92a19fc888e8076d0b3fbd4b9dc96a0a" containerID="702f805b25c6f08786120ec949d1aadd3b3aa8077f522af5f066641dbedef73e" exitCode=0 Jul 20 12:44:22.589980 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:22.589950 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" event={"ID":"92a19fc888e8076d0b3fbd4b9dc96a0a","Type":"ContainerDied","Data":"702f805b25c6f08786120ec949d1aadd3b3aa8077f522af5f066641dbedef73e"} Jul 20 12:44:23.517198 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:23.517157 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:23.517746 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:23.517314 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:23.607898 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:23.607272 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" event={"ID":"92a19fc888e8076d0b3fbd4b9dc96a0a","Type":"ContainerStarted","Data":"218923c22c8e275fbb55cf24380b698e248c48ca402dc2045d2cb4f91f66d9d5"} Jul 20 12:44:24.093424 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:24.093363 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:24.093649 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:24.093444 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:24.093649 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:24.093581 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:24.093649 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:24.093649 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret podName:1220141b-d9f7-4e87-95e8-cc658b486b7f nodeName:}" failed. No retries permitted until 2026-07-20 12:44:28.093629552 +0000 UTC m=+10.065835370 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret") pod "global-pull-secret-syncer-wvc67" (UID: "1220141b-d9f7-4e87-95e8-cc658b486b7f") : object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:24.094081 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:24.094060 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:24.094162 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:24.094120 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs podName:24ad9090-71b2-4f16-9389-19f115738cf6 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:28.09410333 +0000 UTC m=+10.066309148 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs") pod "network-metrics-daemon-6gkrz" (UID: "24ad9090-71b2-4f16-9389-19f115738cf6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:24.194893 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:24.194847 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbvgj\" (UniqueName: \"kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj\") pod \"network-check-target-67vbc\" (UID: \"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72\") " pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:24.195120 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:24.195089 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 12:44:24.195120 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:24.195112 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 12:44:24.195260 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:24.195126 2570 projected.go:196] Error preparing data for projected volume kube-api-access-nbvgj for pod openshift-network-diagnostics/network-check-target-67vbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:24.195260 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:24.195189 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj podName:6e808ebf-fa3a-423a-a77f-5ae8d13c6e72 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:28.195167998 +0000 UTC m=+10.167373811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nbvgj" (UniqueName: "kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj") pod "network-check-target-67vbc" (UID: "6e808ebf-fa3a-423a-a77f-5ae8d13c6e72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:24.517765 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:24.517631 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:24.518449 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:24.517789 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:24.518449 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:24.518318 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:24.518449 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:24.518424 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:25.516298 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:25.515696 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:25.516298 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:25.515846 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:26.517476 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:26.517117 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:26.517476 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:26.517260 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:26.518052 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:26.517822 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:26.518052 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:26.517940 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:27.516278 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:27.515657 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:27.516278 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:27.515817 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:28.130807 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:28.130770 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:28.131313 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:28.130822 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:28.131313 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:28.130952 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:28.131313 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:28.131034 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret podName:1220141b-d9f7-4e87-95e8-cc658b486b7f nodeName:}" failed. No retries permitted until 2026-07-20 12:44:36.131012899 +0000 UTC m=+18.103218716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret") pod "global-pull-secret-syncer-wvc67" (UID: "1220141b-d9f7-4e87-95e8-cc658b486b7f") : object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:28.131313 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:28.130953 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:28.131313 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:28.131119 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs podName:24ad9090-71b2-4f16-9389-19f115738cf6 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:36.131097622 +0000 UTC m=+18.103303450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs") pod "network-metrics-daemon-6gkrz" (UID: "24ad9090-71b2-4f16-9389-19f115738cf6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:28.232221 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:28.232183 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbvgj\" (UniqueName: \"kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj\") pod \"network-check-target-67vbc\" (UID: \"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72\") " pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:28.232454 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:28.232429 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 12:44:28.232454 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:28.232460 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 12:44:28.232454 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:28.232474 2570 projected.go:196] Error preparing data for projected volume kube-api-access-nbvgj for pod openshift-network-diagnostics/network-check-target-67vbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:28.232737 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:28.232537 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj podName:6e808ebf-fa3a-423a-a77f-5ae8d13c6e72 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:36.232518694 +0000 UTC m=+18.204724509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nbvgj" (UniqueName: "kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj") pod "network-check-target-67vbc" (UID: "6e808ebf-fa3a-423a-a77f-5ae8d13c6e72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:28.517164 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:28.517076 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:28.517164 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:28.517080 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:28.517419 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:28.517195 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:28.517419 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:28.517253 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:29.516131 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:29.516087 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:29.516614 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:29.516246 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:30.515354 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:30.515158 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:30.515354 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:30.515299 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:30.515877 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:30.515719 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:30.515877 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:30.515834 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:31.515644 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:31.515605 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:31.516098 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:31.515741 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:32.515097 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:32.515073 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:32.515197 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:32.515180 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:32.515290 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:32.515272 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:32.515417 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:32.515398 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:33.515499 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:33.515454 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:33.515957 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:33.515578 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:34.515865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:34.515829 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:34.516319 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:34.515838 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:34.516319 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:34.515968 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:34.516319 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:34.516047 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:35.515070 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:35.515040 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:35.515344 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:35.515151 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:36.194657 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:36.194619 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:36.195045 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:36.194670 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:36.195045 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:36.194768 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:36.195045 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:36.194831 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret podName:1220141b-d9f7-4e87-95e8-cc658b486b7f nodeName:}" failed. No retries permitted until 2026-07-20 12:44:52.194816295 +0000 UTC m=+34.167022105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret") pod "global-pull-secret-syncer-wvc67" (UID: "1220141b-d9f7-4e87-95e8-cc658b486b7f") : object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:36.195045 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:36.194768 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:36.195045 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:36.194902 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs podName:24ad9090-71b2-4f16-9389-19f115738cf6 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:52.194888459 +0000 UTC m=+34.167094268 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs") pod "network-metrics-daemon-6gkrz" (UID: "24ad9090-71b2-4f16-9389-19f115738cf6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:36.295457 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:36.295421 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbvgj\" (UniqueName: \"kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj\") pod \"network-check-target-67vbc\" (UID: \"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72\") " pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:36.295633 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:36.295591 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 12:44:36.295633 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:36.295614 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 12:44:36.295633 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:36.295625 2570 projected.go:196] Error preparing data for projected volume kube-api-access-nbvgj for pod openshift-network-diagnostics/network-check-target-67vbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:36.295744 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:36.295680 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj podName:6e808ebf-fa3a-423a-a77f-5ae8d13c6e72 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:52.295663309 +0000 UTC m=+34.267869138 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nbvgj" (UniqueName: "kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj") pod "network-check-target-67vbc" (UID: "6e808ebf-fa3a-423a-a77f-5ae8d13c6e72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:36.515358 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:36.515266 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:36.515522 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:36.515420 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:36.515522 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:36.515477 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:36.515637 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:36.515582 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:37.515088 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:37.515054 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:37.515516 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:37.515161 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:38.518076 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:38.518048 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:38.518533 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:38.518161 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:38.518533 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:38.518225 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:38.518533 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:38.518334 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:39.515873 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:39.515839 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:39.516049 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:39.515960 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:40.518648 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:40.518608 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:40.519109 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:40.518609 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:40.519109 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:40.518738 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:40.519109 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:40.518806 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:41.516064 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.515667 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:41.516260 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:41.516122 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:41.649985 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.649944 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" event={"ID":"1910d8dc-faf0-408f-8639-b05dc8bb0769","Type":"ContainerStarted","Data":"75c5b3a2e6cb68f0104a007b52fd3a6206e8870828ff52ebdb52d30208972290"} Jul 20 12:44:41.651263 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.651238 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vs9qv" event={"ID":"fd720144-9924-46ef-a880-0c196604451b","Type":"ContainerStarted","Data":"a6899bdcd08c4c4be1b56e75839f291ba854326acd39027752b0169435034515"} Jul 20 12:44:41.652638 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.652610 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kpsm8" event={"ID":"717fdc62-4efb-490e-bd34-e147ac34ddbf","Type":"ContainerStarted","Data":"3ad6e96214fab2da1a2e88cee94d1105a18d2f782eebbb5cfe50a325b80f9f40"} Jul 20 12:44:41.654051 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.654028 2570 generic.go:356] "Generic (PLEG): container finished" podID="fc1cf019-6b91-49b4-b73a-1c41a257269d" containerID="91cc17282612e44584746ef808fbe6f9d780a750fe6996600615502af499013c" exitCode=0 Jul 20 12:44:41.654177 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.654096 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhm8h" event={"ID":"fc1cf019-6b91-49b4-b73a-1c41a257269d","Type":"ContainerDied","Data":"91cc17282612e44584746ef808fbe6f9d780a750fe6996600615502af499013c"} Jul 20 12:44:41.655767 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.655513 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jstg9" event={"ID":"c1fc1557-eab5-4747-9378-59a27b43a32a","Type":"ContainerStarted","Data":"271c58e8bd6db2a992abcdc12422a5c3fd51bf549added3eaaa5bae820f734ec"} Jul 20 12:44:41.657151 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.657131 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-55hwm" event={"ID":"36c5fc1a-f0e9-4443-b922-cd84c06112e5","Type":"ContainerStarted","Data":"c6362fd8d46a6d758b36e677463eeb0c7643b058b21f576f856a360fbce8d6ea"} Jul 20 12:44:41.660880 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.660772 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" event={"ID":"9eb5f9c7-9a60-4075-b022-e5308b67d79c","Type":"ContainerStarted","Data":"85b9373f0809c6021d4e5e6169b510288e37c69bfa4f1b4d8ac5e7cb391cb2c3"} Jul 20 12:44:41.660880 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.660862 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" event={"ID":"9eb5f9c7-9a60-4075-b022-e5308b67d79c","Type":"ContainerStarted","Data":"0617c7ca1cff5584fc3e359b9890cbeb5d19620b96ae9d2994039aebf511a0f4"} Jul 20 12:44:41.661083 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.660910 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" event={"ID":"9eb5f9c7-9a60-4075-b022-e5308b67d79c","Type":"ContainerStarted","Data":"979b2525b51ec319e5350f156434b387bafb1e3aac3cbdcaf7fe06fa70981023"} Jul 20 12:44:41.661083 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.660955 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" event={"ID":"9eb5f9c7-9a60-4075-b022-e5308b67d79c","Type":"ContainerStarted","Data":"710b530cd45b2ebb4a546c0c4dfc4cc0adc101f2dd37dc937329ef48c9b23814"} Jul 20 12:44:41.661083 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.661003 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" event={"ID":"9eb5f9c7-9a60-4075-b022-e5308b67d79c","Type":"ContainerStarted","Data":"b49d2657f6206f8e8a65fd1497b385aa0578e7e652b93f44645d51c0bc7e90c3"} Jul 20 12:44:41.661083 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.661046 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" event={"ID":"9eb5f9c7-9a60-4075-b022-e5308b67d79c","Type":"ContainerStarted","Data":"39c8e9c59a921704fb2ecd21f5c8071285d06e4db71307ddad20ca3698e6b571"} Jul 20 12:44:41.662541 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.662515 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8k96c" event={"ID":"c9470643-a690-4466-b8ec-496ccf18c5e6","Type":"ContainerStarted","Data":"c78d3db84add4f99b89afd2528d862306472d5b7cfaae01077b86fec303aaadd"} Jul 20 12:44:41.666607 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.666561 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" podStartSLOduration=21.666547458 podStartE2EDuration="21.666547458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:44:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:23.626423855 +0000 UTC m=+5.598629692" watchObservedRunningTime="2026-07-20 12:44:41.666547458 +0000 UTC m=+23.638753290" Jul 20 12:44:41.684078 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.682825 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vs9qv" podStartSLOduration=3.438419865 podStartE2EDuration="22.682802684s" totalImagesPullingTime="19.244382819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:41.666268504 +0000 UTC m=+23.638474335" watchObservedRunningTime="2026-07-20 12:44:41.682802684 +0000 UTC m=+23.655008519" Jul 20 12:44:41.684078 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.683324 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-55hwm" podStartSLOduration=11.491662359 podStartE2EDuration="22.683315401s" totalImagesPullingTime="11.191653042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:41.682145491 +0000 UTC m=+23.654351323" watchObservedRunningTime="2026-07-20 12:44:41.683315401 +0000 UTC m=+23.655521234" Jul 20 12:44:41.721483 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.721427 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kpsm8" podStartSLOduration=3.415688877 podStartE2EDuration="22.721410839s" totalImagesPullingTime="19.305721962s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:41.720980007 +0000 UTC m=+23.693185839" watchObservedRunningTime="2026-07-20 12:44:41.721410839 +0000 UTC m=+23.693616670" Jul 20 12:44:41.736477 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.736416 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8k96c" podStartSLOduration=3.494204349 podStartE2EDuration="22.736396117s" totalImagesPullingTime="19.242191768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:41.736269981 +0000 UTC m=+23.708475814" watchObservedRunningTime="2026-07-20 12:44:41.736396117 +0000 UTC m=+23.708601951" Jul 20 12:44:41.750431 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:41.750358 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jstg9" podStartSLOduration=3.511824157 podStartE2EDuration="22.750340063s" totalImagesPullingTime="19.238515906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:41.749823352 +0000 UTC m=+23.722029184" watchObservedRunningTime="2026-07-20 12:44:41.750340063 +0000 UTC m=+23.722545895" Jul 20 12:44:42.028915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:42.028881 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 12:44:42.515310 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:42.515278 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:42.515813 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:42.515289 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:42.515813 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:42.515428 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:42.515813 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:42.515525 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:42.531544 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:42.531436 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T12:44:42.028907016Z","UUID":"54203bcc-6a4e-4962-9c28-46c370902922","Handler":null,"Name":"","Endpoint":""} Jul 20 12:44:42.533290 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:42.533264 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 12:44:42.533443 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:42.533313 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 12:44:42.665943 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:42.665906 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hx52w" event={"ID":"29168ce6-87e8-41ea-aa38-d9e184ffb90b","Type":"ContainerStarted","Data":"30e1e1438b264a8c3c2058af1773b9fc54b09004df33ff025a00b505ab3b7a59"} Jul 20 12:44:42.667933 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:42.667899 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" event={"ID":"1910d8dc-faf0-408f-8639-b05dc8bb0769","Type":"ContainerStarted","Data":"3eaee0b010eee0be238923e91a579e367a4bd100f3b41cdcbfde51b9421341c7"} Jul 20 12:44:42.682302 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:42.682248 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hx52w" podStartSLOduration=4.442324723 podStartE2EDuration="23.682226158s" totalImagesPullingTime="19.239901435s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:42.682155357 +0000 UTC m=+24.654361183" watchObservedRunningTime="2026-07-20 12:44:42.682226158 +0000 UTC m=+24.654431995" Jul 20 12:44:43.516119 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:43.516083 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:43.516428 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:43.516222 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:43.674161 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:43.674066 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" event={"ID":"9eb5f9c7-9a60-4075-b022-e5308b67d79c","Type":"ContainerStarted","Data":"8ed144ead1c7eb29153f1a364939c56b61886a65e02b2830ea6c4e0bb61bc19e"} Jul 20 12:44:43.676329 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:43.676292 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" event={"ID":"1910d8dc-faf0-408f-8639-b05dc8bb0769","Type":"ContainerStarted","Data":"5caaa297947afc9b7a792d4d4f385302dd6834300e254771cc7bb7d867b9c5e8"} Jul 20 12:44:43.694432 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:43.694357 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t6zdn" podStartSLOduration=3.259700658 podStartE2EDuration="24.694338655s" totalImagesPullingTime="21.434637997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:43.694164018 +0000 UTC m=+25.666369843" watchObservedRunningTime="2026-07-20 12:44:43.694338655 +0000 UTC m=+25.666544488" Jul 20 12:44:44.212659 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:44.212622 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:44.213315 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:44.213289 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:44.515710 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:44.515622 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:44.515871 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:44.515630 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:44.515871 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:44.515752 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:44.515871 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:44.515826 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:44.678594 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:44.678568 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:44.679183 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:44.679160 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-55hwm" Jul 20 12:44:45.515176 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:45.515140 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:45.515383 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:45.515268 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:46.516262 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:46.516022 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:46.517051 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:46.516087 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:46.517051 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:46.516285 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:46.517051 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:46.516363 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:46.683986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:46.683952 2570 generic.go:356] "Generic (PLEG): container finished" podID="fc1cf019-6b91-49b4-b73a-1c41a257269d" containerID="a82f300f0db6d6fc58784997276bfcce7772fcb1ae9b236b2a01333439cee87a" exitCode=0 Jul 20 12:44:46.684134 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:46.684041 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhm8h" event={"ID":"fc1cf019-6b91-49b4-b73a-1c41a257269d","Type":"ContainerDied","Data":"a82f300f0db6d6fc58784997276bfcce7772fcb1ae9b236b2a01333439cee87a"} Jul 20 12:44:46.687746 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:46.687715 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" event={"ID":"9eb5f9c7-9a60-4075-b022-e5308b67d79c","Type":"ContainerStarted","Data":"1d627cd8b1c2831399c27d109e2a49707e960226c08542b0045ada309a7fddca"} Jul 20 12:44:46.726016 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:46.725958 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" podStartSLOduration=8.403829018 podStartE2EDuration="27.725940679s" totalImagesPullingTime="19.322111661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:46.725500851 +0000 UTC m=+28.697706682" watchObservedRunningTime="2026-07-20 12:44:46.725940679 +0000 UTC m=+28.698146511" Jul 20 12:44:47.515296 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.515097 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:47.515531 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:47.515407 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:47.691261 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.691115 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:47.691261 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.691202 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:47.691812 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.691430 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:47.716563 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.716533 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:47.716747 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.716679 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:44:47.940995 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.940951 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wvc67"] Jul 20 12:44:47.941159 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.941051 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:47.941159 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:47.941136 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:47.943769 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.943739 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6gkrz"] Jul 20 12:44:47.943928 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.943874 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:47.944019 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:47.943996 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:47.958316 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.958281 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-67vbc"] Jul 20 12:44:47.958495 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:47.958425 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:47.958552 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:47.958529 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:48.694434 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:48.694393 2570 generic.go:356] "Generic (PLEG): container finished" podID="fc1cf019-6b91-49b4-b73a-1c41a257269d" containerID="0ef5f1602348fc79d2468dd5e5925f30bd6d5b9640e4f6f98559c5bbb90f319e" exitCode=0 Jul 20 12:44:48.694925 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:48.694474 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhm8h" event={"ID":"fc1cf019-6b91-49b4-b73a-1c41a257269d","Type":"ContainerDied","Data":"0ef5f1602348fc79d2468dd5e5925f30bd6d5b9640e4f6f98559c5bbb90f319e"} Jul 20 12:44:49.515817 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:49.515783 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:49.515981 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:49.515795 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:49.515981 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:49.515936 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:49.515981 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:49.515797 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:49.515981 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:49.515967 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:49.516138 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:49.516038 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:50.701414 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:50.701195 2570 generic.go:356] "Generic (PLEG): container finished" podID="fc1cf019-6b91-49b4-b73a-1c41a257269d" containerID="f89fc67568f9719ee27d187ba82c00f7f022f55db72b5287344e5cb70b874d93" exitCode=0 Jul 20 12:44:50.701873 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:50.701279 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhm8h" event={"ID":"fc1cf019-6b91-49b4-b73a-1c41a257269d","Type":"ContainerDied","Data":"f89fc67568f9719ee27d187ba82c00f7f022f55db72b5287344e5cb70b874d93"} Jul 20 12:44:51.515406 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:51.515359 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:51.515605 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:51.515359 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:51.515605 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:51.515499 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvc67" podUID="1220141b-d9f7-4e87-95e8-cc658b486b7f" Jul 20 12:44:51.515605 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:51.515359 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:51.515744 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:51.515558 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67vbc" podUID="6e808ebf-fa3a-423a-a77f-5ae8d13c6e72" Jul 20 12:44:51.515744 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:51.515661 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gkrz" podUID="24ad9090-71b2-4f16-9389-19f115738cf6" Jul 20 12:44:52.219224 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:52.219187 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:52.219873 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:52.219234 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:52.219873 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:52.219337 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:52.219873 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:52.219361 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:52.219873 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:52.219436 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs podName:24ad9090-71b2-4f16-9389-19f115738cf6 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:24.21941273 +0000 UTC m=+66.191618540 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs") pod "network-metrics-daemon-6gkrz" (UID: "24ad9090-71b2-4f16-9389-19f115738cf6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 12:44:52.219873 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:52.219458 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret podName:1220141b-d9f7-4e87-95e8-cc658b486b7f nodeName:}" failed. No retries permitted until 2026-07-20 12:45:24.219447876 +0000 UTC m=+66.191653692 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret") pod "global-pull-secret-syncer-wvc67" (UID: "1220141b-d9f7-4e87-95e8-cc658b486b7f") : object "kube-system"/"original-pull-secret" not registered Jul 20 12:44:52.319909 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:52.319872 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbvgj\" (UniqueName: \"kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj\") pod \"network-check-target-67vbc\" (UID: \"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72\") " pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:52.320092 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:52.320013 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 12:44:52.320092 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:52.320033 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 12:44:52.320092 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:52.320042 2570 projected.go:196] Error preparing data for projected volume kube-api-access-nbvgj for pod openshift-network-diagnostics/network-check-target-67vbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:52.320204 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:52.320100 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj podName:6e808ebf-fa3a-423a-a77f-5ae8d13c6e72 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:24.320085617 +0000 UTC m=+66.292291427 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nbvgj" (UniqueName: "kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj") pod "network-check-target-67vbc" (UID: "6e808ebf-fa3a-423a-a77f-5ae8d13c6e72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 12:44:53.320088 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.320055 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeReady" Jul 20 12:44:53.320639 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.320220 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 12:44:53.359000 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.358962 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tlzx5"] Jul 20 12:44:53.361758 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.361717 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8zhcc"] Jul 20 12:44:53.361916 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.361857 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.363666 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.363643 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8zhcc" Jul 20 12:44:53.365051 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.365029 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 12:44:53.365051 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.365040 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dcf82\"" Jul 20 12:44:53.365674 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.365655 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 12:44:53.365777 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.365694 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 12:44:53.366116 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.365926 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 12:44:53.366116 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.365933 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 12:44:53.366116 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.366016 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-47mnr\"" Jul 20 12:44:53.372270 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.372243 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tlzx5"] Jul 20 12:44:53.377724 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.377657 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8zhcc"] Jul 20 12:44:53.515593 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.515552 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:44:53.515780 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.515610 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:44:53.515780 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.515564 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:44:53.517946 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.517922 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 12:44:53.518088 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.518016 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 12:44:53.518088 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.518044 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 12:44:53.518088 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.518072 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pvfjj\"" Jul 20 12:44:53.518267 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.518139 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cg6n8\"" Jul 20 12:44:53.518267 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.518170 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 12:44:53.528190 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.528155 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.528332 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.528210 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8319b26b-2857-4f07-aeae-9116dbb18e10-config-volume\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.528332 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.528287 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8319b26b-2857-4f07-aeae-9116dbb18e10-tmp-dir\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.528332 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.528323 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7s7z\" (UniqueName: \"kubernetes.io/projected/8319b26b-2857-4f07-aeae-9116dbb18e10-kube-api-access-b7s7z\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.528508 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.528385 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f09b4592-c85d-4f65-b6d1-63e0d175afa4-cert\") pod \"ingress-canary-8zhcc\" (UID: \"f09b4592-c85d-4f65-b6d1-63e0d175afa4\") " pod="openshift-ingress-canary/ingress-canary-8zhcc" Jul 20 12:44:53.528508 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.528412 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vf89\" (UniqueName: \"kubernetes.io/projected/f09b4592-c85d-4f65-b6d1-63e0d175afa4-kube-api-access-6vf89\") pod \"ingress-canary-8zhcc\" (UID: \"f09b4592-c85d-4f65-b6d1-63e0d175afa4\") " pod="openshift-ingress-canary/ingress-canary-8zhcc" Jul 20 12:44:53.629147 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.629068 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.629147 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.629114 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8319b26b-2857-4f07-aeae-9116dbb18e10-config-volume\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.629361 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.629187 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8319b26b-2857-4f07-aeae-9116dbb18e10-tmp-dir\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.629361 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:53.629294 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 12:44:53.629361 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.629331 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7s7z\" (UniqueName: \"kubernetes.io/projected/8319b26b-2857-4f07-aeae-9116dbb18e10-kube-api-access-b7s7z\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.629539 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:53.629445 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls podName:8319b26b-2857-4f07-aeae-9116dbb18e10 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:54.129419607 +0000 UTC m=+36.101625482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls") pod "dns-default-tlzx5" (UID: "8319b26b-2857-4f07-aeae-9116dbb18e10") : secret "dns-default-metrics-tls" not found Jul 20 12:44:53.629539 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.629526 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f09b4592-c85d-4f65-b6d1-63e0d175afa4-cert\") pod \"ingress-canary-8zhcc\" (UID: \"f09b4592-c85d-4f65-b6d1-63e0d175afa4\") " pod="openshift-ingress-canary/ingress-canary-8zhcc" Jul 20 12:44:53.629641 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.629563 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vf89\" (UniqueName: \"kubernetes.io/projected/f09b4592-c85d-4f65-b6d1-63e0d175afa4-kube-api-access-6vf89\") pod \"ingress-canary-8zhcc\" (UID: \"f09b4592-c85d-4f65-b6d1-63e0d175afa4\") " pod="openshift-ingress-canary/ingress-canary-8zhcc" Jul 20 12:44:53.629697 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.629638 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8319b26b-2857-4f07-aeae-9116dbb18e10-tmp-dir\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.629780 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.629749 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8319b26b-2857-4f07-aeae-9116dbb18e10-config-volume\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.634042 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.634021 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f09b4592-c85d-4f65-b6d1-63e0d175afa4-cert\") pod \"ingress-canary-8zhcc\" (UID: \"f09b4592-c85d-4f65-b6d1-63e0d175afa4\") " pod="openshift-ingress-canary/ingress-canary-8zhcc" Jul 20 12:44:53.637621 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.637537 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7s7z\" (UniqueName: \"kubernetes.io/projected/8319b26b-2857-4f07-aeae-9116dbb18e10-kube-api-access-b7s7z\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:53.637755 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.637728 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vf89\" (UniqueName: \"kubernetes.io/projected/f09b4592-c85d-4f65-b6d1-63e0d175afa4-kube-api-access-6vf89\") pod \"ingress-canary-8zhcc\" (UID: \"f09b4592-c85d-4f65-b6d1-63e0d175afa4\") " pod="openshift-ingress-canary/ingress-canary-8zhcc" Jul 20 12:44:53.685731 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:53.685691 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8zhcc" Jul 20 12:44:54.134058 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:54.134025 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:54.134254 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:54.134162 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 12:44:54.134254 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:54.134225 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls podName:8319b26b-2857-4f07-aeae-9116dbb18e10 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:55.13421067 +0000 UTC m=+37.106416498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls") pod "dns-default-tlzx5" (UID: "8319b26b-2857-4f07-aeae-9116dbb18e10") : secret "dns-default-metrics-tls" not found Jul 20 12:44:54.728789 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:54.728556 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8zhcc"] Jul 20 12:44:54.734331 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:44:54.734304 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf09b4592_c85d_4f65_b6d1_63e0d175afa4.slice/crio-c304e433af724b386041ddeb6bb3b12e60ce3404da55051773e74c20f02a2de9 WatchSource:0}: Error finding container c304e433af724b386041ddeb6bb3b12e60ce3404da55051773e74c20f02a2de9: Status 404 returned error can't find the container with id c304e433af724b386041ddeb6bb3b12e60ce3404da55051773e74c20f02a2de9 Jul 20 12:44:54.750588 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:54.750543 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:44:54.837422 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:54.837352 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:44:55.141283 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:55.141238 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:55.141487 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:55.141393 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 12:44:55.141487 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:55.141449 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls podName:8319b26b-2857-4f07-aeae-9116dbb18e10 nodeName:}" failed. No retries permitted until 2026-07-20 12:44:57.141434812 +0000 UTC m=+39.113640622 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls") pod "dns-default-tlzx5" (UID: "8319b26b-2857-4f07-aeae-9116dbb18e10") : secret "dns-default-metrics-tls" not found Jul 20 12:44:55.273002 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:55.272964 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:44:55.323713 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:55.323672 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:44:55.718048 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:55.718009 2570 generic.go:356] "Generic (PLEG): container finished" podID="fc1cf019-6b91-49b4-b73a-1c41a257269d" containerID="b1a601b13d611cbb3bfb5e01996c866027e0e82443420c0c95bbba0d480c096b" exitCode=0 Jul 20 12:44:55.718251 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:55.718094 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhm8h" event={"ID":"fc1cf019-6b91-49b4-b73a-1c41a257269d","Type":"ContainerDied","Data":"b1a601b13d611cbb3bfb5e01996c866027e0e82443420c0c95bbba0d480c096b"} Jul 20 12:44:55.719562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:55.719399 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8zhcc" event={"ID":"f09b4592-c85d-4f65-b6d1-63e0d175afa4","Type":"ContainerStarted","Data":"c304e433af724b386041ddeb6bb3b12e60ce3404da55051773e74c20f02a2de9"} Jul 20 12:44:56.726359 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:56.726325 2570 generic.go:356] "Generic (PLEG): container finished" podID="fc1cf019-6b91-49b4-b73a-1c41a257269d" containerID="80b0f4f9f8f883621eb9ca6060d0e732fe71f817d5ae43bd97462eccdecdbf78" exitCode=0 Jul 20 12:44:56.726787 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:56.726392 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhm8h" event={"ID":"fc1cf019-6b91-49b4-b73a-1c41a257269d","Type":"ContainerDied","Data":"80b0f4f9f8f883621eb9ca6060d0e732fe71f817d5ae43bd97462eccdecdbf78"} Jul 20 12:44:57.157789 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:57.157598 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:44:57.157789 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:57.157742 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 12:44:57.157992 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:44:57.157839 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls podName:8319b26b-2857-4f07-aeae-9116dbb18e10 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:01.15782105 +0000 UTC m=+43.130026860 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls") pod "dns-default-tlzx5" (UID: "8319b26b-2857-4f07-aeae-9116dbb18e10") : secret "dns-default-metrics-tls" not found Jul 20 12:44:57.730434 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:57.730309 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8zhcc" event={"ID":"f09b4592-c85d-4f65-b6d1-63e0d175afa4","Type":"ContainerStarted","Data":"53f05b5c221e5902425f40cf1d26090d6e699905728217622b33857dcf09edc1"} Jul 20 12:44:57.733454 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:57.733425 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhm8h" event={"ID":"fc1cf019-6b91-49b4-b73a-1c41a257269d","Type":"ContainerStarted","Data":"a6509fcdf98dabf2ca7b0050708c088b9aa826760eff7baf4534867e92918bf9"} Jul 20 12:44:57.745080 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:57.745029 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8zhcc" podStartSLOduration=2.124112193 podStartE2EDuration="4.745012049s" totalImagesPullingTime="2.620899856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:57.744416347 +0000 UTC m=+39.716622179" watchObservedRunningTime="2026-07-20 12:44:57.745012049 +0000 UTC m=+39.717217880" Jul 20 12:44:57.764021 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:57.763959 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lhm8h" podStartSLOduration=9.378392537 podStartE2EDuration="38.763936392s" totalImagesPullingTime="29.385543855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:44:57.763142852 +0000 UTC m=+39.735348682" watchObservedRunningTime="2026-07-20 12:44:57.763936392 +0000 UTC m=+39.736142224" Jul 20 12:44:57.883814 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:57.883781 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:44:57.984740 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:57.984657 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:44:58.676420 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:58.676388 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:44:58.740660 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:44:58.740630 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:00.472552 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.472520 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-xkbwk"] Jul 20 12:45:00.504858 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.504820 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-cwqb4"] Jul 20 12:45:00.505019 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.504932 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xkbwk" Jul 20 12:45:00.506844 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.506822 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tg925\"" Jul 20 12:45:00.518792 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.518765 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.520756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.520730 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 20 12:45:00.521068 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.521045 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7mzd6\"" Jul 20 12:45:00.521191 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.521073 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 20 12:45:00.521191 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.521166 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 20 12:45:00.521304 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.521203 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 20 12:45:00.522987 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.522965 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-594bbb7854-524rq"] Jul 20 12:45:00.526095 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.526072 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 20 12:45:00.545442 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.545410 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-xkbwk"] Jul 20 12:45:00.545442 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.545450 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-cwqb4"] Jul 20 12:45:00.545647 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.545466 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-594bbb7854-524rq"] Jul 20 12:45:00.545647 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.545552 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.547403 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.547380 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jkxmq\"" Jul 20 12:45:00.547525 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.547387 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 20 12:45:00.547525 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.547443 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 20 12:45:00.547750 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.547734 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 20 12:45:00.548010 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.547977 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 12:45:00.548010 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.547996 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 12:45:00.548145 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.547999 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 20 12:45:00.581877 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.581842 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq"] Jul 20 12:45:00.596784 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.596751 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-rqn64"] Jul 20 12:45:00.596984 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.596889 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" Jul 20 12:45:00.602534 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.602503 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 20 12:45:00.603685 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.603655 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 20 12:45:00.603838 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.603819 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-knjkf\"" Jul 20 12:45:00.604060 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.604042 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 20 12:45:00.604060 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.604051 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 20 12:45:00.614817 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.614788 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6fcfdc44d-flp46"] Jul 20 12:45:00.614955 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.614902 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-rqn64" Jul 20 12:45:00.620026 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.620003 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 20 12:45:00.620497 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.620470 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 20 12:45:00.621393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.621305 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-g5tcl\"" Jul 20 12:45:00.632875 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.632843 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq"] Jul 20 12:45:00.632875 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.632887 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-rqn64"] Jul 20 12:45:00.633065 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.632961 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.635279 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.635260 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 20 12:45:00.635829 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.635812 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 20 12:45:00.636058 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.636040 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 20 12:45:00.636393 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.636378 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4nbld\"" Jul 20 12:45:00.649004 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.648978 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 20 12:45:00.651185 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.651109 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fcfdc44d-flp46"] Jul 20 12:45:00.676323 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.676279 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:00.682717 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.682666 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.682878 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.682725 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/071ec3e8-c145-4cb7-9ab5-116975ec402c-serving-cert\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.682878 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.682753 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.682878 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.682777 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/071ec3e8-c145-4cb7-9ab5-116975ec402c-trusted-ca\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.682878 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.682850 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/071ec3e8-c145-4cb7-9ab5-116975ec402c-config\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.683027 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.682915 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n5w6\" (UniqueName: \"kubernetes.io/projected/6728fc86-fff5-4df8-8231-626b9b01e80d-kube-api-access-9n5w6\") pod \"network-check-source-7bc5b6bf85-xkbwk\" (UID: \"6728fc86-fff5-4df8-8231-626b9b01e80d\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xkbwk" Jul 20 12:45:00.683027 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.682946 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-462zs\" (UniqueName: \"kubernetes.io/projected/071ec3e8-c145-4cb7-9ab5-116975ec402c-kube-api-access-462zs\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.683027 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.682977 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb467\" (UniqueName: \"kubernetes.io/projected/e66b1363-7be7-4cad-ad86-053fbe8c5596-kube-api-access-cb467\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.683027 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.683002 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-default-certificate\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.683143 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.683031 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-stats-auth\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.737565 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.737447 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:00.784238 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784199 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-installation-pull-secrets\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.784238 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784239 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.784480 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784261 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-image-registry-private-configuration\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.784480 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:00.784412 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:01.284391283 +0000 UTC m=+43.256597093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : configmap references non-existent config key: service-ca.crt Jul 20 12:45:00.784480 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784448 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47hjg\" (UniqueName: \"kubernetes.io/projected/72ab29c8-e484-4ffa-b85b-48848df8d63c-kube-api-access-47hjg\") pod \"volume-data-source-validator-78cf4c6f6-rqn64\" (UID: \"72ab29c8-e484-4ffa-b85b-48848df8d63c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-rqn64" Jul 20 12:45:00.784480 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784469 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.784608 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784514 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/071ec3e8-c145-4cb7-9ab5-116975ec402c-trusted-ca\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.784608 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784539 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-certificates\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.784608 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784561 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69lh6\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-kube-api-access-69lh6\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.784697 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784608 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/071ec3e8-c145-4cb7-9ab5-116975ec402c-config\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.784697 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784658 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n5w6\" (UniqueName: \"kubernetes.io/projected/6728fc86-fff5-4df8-8231-626b9b01e80d-kube-api-access-9n5w6\") pod \"network-check-source-7bc5b6bf85-xkbwk\" (UID: \"6728fc86-fff5-4df8-8231-626b9b01e80d\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xkbwk" Jul 20 12:45:00.784697 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784681 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-462zs\" (UniqueName: \"kubernetes.io/projected/071ec3e8-c145-4cb7-9ab5-116975ec402c-kube-api-access-462zs\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.784781 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784736 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb467\" (UniqueName: \"kubernetes.io/projected/e66b1363-7be7-4cad-ad86-053fbe8c5596-kube-api-access-cb467\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.784781 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784762 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-stats-auth\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.784853 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784789 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-trusted-ca\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.784853 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784814 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-zwnsq\" (UID: \"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" Jul 20 12:45:00.784853 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784840 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-zwnsq\" (UID: \"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" Jul 20 12:45:00.784970 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784867 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-ca-trust-extracted\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.784970 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784891 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-bound-sa-token\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.784970 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784917 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.784970 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.784963 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/071ec3e8-c145-4cb7-9ab5-116975ec402c-serving-cert\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.785156 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:00.785054 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 12:45:00.785156 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:00.785115 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:01.285099246 +0000 UTC m=+43.257305056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : secret "router-metrics-certs-default" not found Jul 20 12:45:00.785260 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.785180 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6xlb\" (UniqueName: \"kubernetes.io/projected/ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca-kube-api-access-p6xlb\") pod \"kube-storage-version-migrator-operator-7658d9b749-zwnsq\" (UID: \"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" Jul 20 12:45:00.785260 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.785239 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-default-certificate\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.785571 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.785546 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/071ec3e8-c145-4cb7-9ab5-116975ec402c-config\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.785707 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.785680 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/071ec3e8-c145-4cb7-9ab5-116975ec402c-trusted-ca\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.788888 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.788857 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/071ec3e8-c145-4cb7-9ab5-116975ec402c-serving-cert\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.788991 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.788908 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-stats-auth\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.788991 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.788917 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-default-certificate\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.792878 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.792856 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb467\" (UniqueName: \"kubernetes.io/projected/e66b1363-7be7-4cad-ad86-053fbe8c5596-kube-api-access-cb467\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:00.793068 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.793045 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-462zs\" (UniqueName: \"kubernetes.io/projected/071ec3e8-c145-4cb7-9ab5-116975ec402c-kube-api-access-462zs\") pod \"console-operator-565b7b569d-cwqb4\" (UID: \"071ec3e8-c145-4cb7-9ab5-116975ec402c\") " pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.793108 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.793082 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n5w6\" (UniqueName: \"kubernetes.io/projected/6728fc86-fff5-4df8-8231-626b9b01e80d-kube-api-access-9n5w6\") pod \"network-check-source-7bc5b6bf85-xkbwk\" (UID: \"6728fc86-fff5-4df8-8231-626b9b01e80d\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xkbwk" Jul 20 12:45:00.814124 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.814074 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xkbwk" Jul 20 12:45:00.828329 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.828295 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:00.886516 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.886429 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-certificates\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.886516 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.886481 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69lh6\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-kube-api-access-69lh6\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.886784 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.886623 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-trusted-ca\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.886784 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.886659 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-zwnsq\" (UID: \"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" Jul 20 12:45:00.886784 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.886696 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-zwnsq\" (UID: \"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" Jul 20 12:45:00.886784 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.886733 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-ca-trust-extracted\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.886784 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.886775 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-bound-sa-token\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.887026 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.886872 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6xlb\" (UniqueName: \"kubernetes.io/projected/ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca-kube-api-access-p6xlb\") pod \"kube-storage-version-migrator-operator-7658d9b749-zwnsq\" (UID: \"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" Jul 20 12:45:00.887026 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.886932 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-installation-pull-secrets\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.887026 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.886968 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-image-registry-private-configuration\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.887026 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.887002 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47hjg\" (UniqueName: \"kubernetes.io/projected/72ab29c8-e484-4ffa-b85b-48848df8d63c-kube-api-access-47hjg\") pod \"volume-data-source-validator-78cf4c6f6-rqn64\" (UID: \"72ab29c8-e484-4ffa-b85b-48848df8d63c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-rqn64" Jul 20 12:45:00.887515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.887030 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.887515 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:00.887155 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 12:45:00.887515 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:00.887170 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fcfdc44d-flp46: secret "image-registry-tls" not found Jul 20 12:45:00.887515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.887171 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-zwnsq\" (UID: \"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" Jul 20 12:45:00.887515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.887184 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-certificates\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.887515 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:00.887221 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls podName:24c7d17a-2831-40f6-a1cf-a07bb02fc3e0 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:01.387204797 +0000 UTC m=+43.359410627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls") pod "image-registry-6fcfdc44d-flp46" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0") : secret "image-registry-tls" not found Jul 20 12:45:00.887876 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.887827 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-ca-trust-extracted\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.888390 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.888331 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-trusted-ca\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.890043 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.889989 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-zwnsq\" (UID: \"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" Jul 20 12:45:00.890161 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.890072 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-installation-pull-secrets\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.890608 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.890584 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-image-registry-private-configuration\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.894357 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.894335 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69lh6\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-kube-api-access-69lh6\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.895466 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.895430 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-bound-sa-token\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:00.895613 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.895581 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47hjg\" (UniqueName: \"kubernetes.io/projected/72ab29c8-e484-4ffa-b85b-48848df8d63c-kube-api-access-47hjg\") pod \"volume-data-source-validator-78cf4c6f6-rqn64\" (UID: \"72ab29c8-e484-4ffa-b85b-48848df8d63c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-rqn64" Jul 20 12:45:00.895709 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.895678 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6xlb\" (UniqueName: \"kubernetes.io/projected/ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca-kube-api-access-p6xlb\") pod \"kube-storage-version-migrator-operator-7658d9b749-zwnsq\" (UID: \"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" Jul 20 12:45:00.911675 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.911628 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" Jul 20 12:45:00.925182 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.925143 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-rqn64" Jul 20 12:45:00.988438 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:00.988267 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-xkbwk"] Jul 20 12:45:00.991730 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:00.991660 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6728fc86_fff5_4df8_8231_626b9b01e80d.slice/crio-94387ce9cb63c76f2f3bc21c3e25d8bbc23ddf110e57e238cbedef8d00f78854 WatchSource:0}: Error finding container 94387ce9cb63c76f2f3bc21c3e25d8bbc23ddf110e57e238cbedef8d00f78854: Status 404 returned error can't find the container with id 94387ce9cb63c76f2f3bc21c3e25d8bbc23ddf110e57e238cbedef8d00f78854 Jul 20 12:45:01.019677 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:01.019643 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-cwqb4"] Jul 20 12:45:01.022513 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:01.022466 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071ec3e8_c145_4cb7_9ab5_116975ec402c.slice/crio-79d912eb09abc9d010d74d3659d38b36c982beed0e5ff7225cdf18f626b29e42 WatchSource:0}: Error finding container 79d912eb09abc9d010d74d3659d38b36c982beed0e5ff7225cdf18f626b29e42: Status 404 returned error can't find the container with id 79d912eb09abc9d010d74d3659d38b36c982beed0e5ff7225cdf18f626b29e42 Jul 20 12:45:01.083533 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:01.083499 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq"] Jul 20 12:45:01.086457 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:01.086430 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4d1035_f1b8_440d_bb30_a0ab1d8ea0ca.slice/crio-f936414958d7f16cdcc2bdaa95f87821cdf85239a9f96c4cd41094f5e13c876c WatchSource:0}: Error finding container f936414958d7f16cdcc2bdaa95f87821cdf85239a9f96c4cd41094f5e13c876c: Status 404 returned error can't find the container with id f936414958d7f16cdcc2bdaa95f87821cdf85239a9f96c4cd41094f5e13c876c Jul 20 12:45:01.107669 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:01.107634 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-rqn64"] Jul 20 12:45:01.110849 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:01.110823 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ab29c8_e484_4ffa_b85b_48848df8d63c.slice/crio-006b30f64be9df5351abd88e51d694c873160281c28eef5e1db4a6482dcdd5c4 WatchSource:0}: Error finding container 006b30f64be9df5351abd88e51d694c873160281c28eef5e1db4a6482dcdd5c4: Status 404 returned error can't find the container with id 006b30f64be9df5351abd88e51d694c873160281c28eef5e1db4a6482dcdd5c4 Jul 20 12:45:01.189266 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:01.189226 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:45:01.189468 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:01.189400 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 12:45:01.189468 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:01.189464 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls podName:8319b26b-2857-4f07-aeae-9116dbb18e10 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:09.189447859 +0000 UTC m=+51.161653669 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls") pod "dns-default-tlzx5" (UID: "8319b26b-2857-4f07-aeae-9116dbb18e10") : secret "dns-default-metrics-tls" not found Jul 20 12:45:01.290695 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:01.290597 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:01.290823 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:01.290706 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:01.290823 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:01.290757 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 12:45:01.290823 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:01.290822 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:02.29080585 +0000 UTC m=+44.263011663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : secret "router-metrics-certs-default" not found Jul 20 12:45:01.290943 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:01.290837 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:02.29083089 +0000 UTC m=+44.263036700 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : configmap references non-existent config key: service-ca.crt Jul 20 12:45:01.391299 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:01.391261 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:01.391473 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:01.391386 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 12:45:01.391473 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:01.391399 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fcfdc44d-flp46: secret "image-registry-tls" not found Jul 20 12:45:01.391473 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:01.391448 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls podName:24c7d17a-2831-40f6-a1cf-a07bb02fc3e0 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:02.391434801 +0000 UTC m=+44.363640610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls") pod "image-registry-6fcfdc44d-flp46" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0") : secret "image-registry-tls" not found Jul 20 12:45:01.747187 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:01.747153 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-rqn64" event={"ID":"72ab29c8-e484-4ffa-b85b-48848df8d63c","Type":"ContainerStarted","Data":"006b30f64be9df5351abd88e51d694c873160281c28eef5e1db4a6482dcdd5c4"} Jul 20 12:45:01.748321 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:01.748298 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xkbwk" event={"ID":"6728fc86-fff5-4df8-8231-626b9b01e80d","Type":"ContainerStarted","Data":"94387ce9cb63c76f2f3bc21c3e25d8bbc23ddf110e57e238cbedef8d00f78854"} Jul 20 12:45:01.749246 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:01.749223 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" event={"ID":"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca","Type":"ContainerStarted","Data":"f936414958d7f16cdcc2bdaa95f87821cdf85239a9f96c4cd41094f5e13c876c"} Jul 20 12:45:01.750103 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:01.750083 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" event={"ID":"071ec3e8-c145-4cb7-9ab5-116975ec402c","Type":"ContainerStarted","Data":"79d912eb09abc9d010d74d3659d38b36c982beed0e5ff7225cdf18f626b29e42"} Jul 20 12:45:02.298335 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:02.298293 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:02.298539 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:02.298420 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:02.298616 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:02.298593 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 12:45:02.298672 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:02.298660 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:04.298640159 +0000 UTC m=+46.270845972 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : secret "router-metrics-certs-default" not found Jul 20 12:45:02.298921 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:02.298879 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:04.298857653 +0000 UTC m=+46.271063468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : configmap references non-existent config key: service-ca.crt Jul 20 12:45:02.399786 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:02.399063 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:02.399786 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:02.399333 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 12:45:02.399786 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:02.399351 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fcfdc44d-flp46: secret "image-registry-tls" not found Jul 20 12:45:02.399786 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:02.399426 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls podName:24c7d17a-2831-40f6-a1cf-a07bb02fc3e0 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:04.399407033 +0000 UTC m=+46.371612862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls") pod "image-registry-6fcfdc44d-flp46" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0") : secret "image-registry-tls" not found Jul 20 12:45:04.316214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:04.316171 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:04.316793 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:04.316264 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:04.316793 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:04.316376 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:08.316340636 +0000 UTC m=+50.288546457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : configmap references non-existent config key: service-ca.crt Jul 20 12:45:04.316793 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:04.316407 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 12:45:04.316793 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:04.316500 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:08.316479599 +0000 UTC m=+50.288685409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : secret "router-metrics-certs-default" not found Jul 20 12:45:04.417618 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:04.417576 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:04.417820 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:04.417728 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 12:45:04.417820 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:04.417744 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fcfdc44d-flp46: secret "image-registry-tls" not found Jul 20 12:45:04.417820 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:04.417807 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls podName:24c7d17a-2831-40f6-a1cf-a07bb02fc3e0 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:08.417788887 +0000 UTC m=+50.389994711 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls") pod "image-registry-6fcfdc44d-flp46" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0") : secret "image-registry-tls" not found Jul 20 12:45:05.797888 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:05.797854 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:05.919228 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:05.919164 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m8gb2"] Jul 20 12:45:05.934054 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:05.934026 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m8gb2"] Jul 20 12:45:05.934209 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:05.934150 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:05.936803 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:05.936778 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 12:45:05.937160 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:05.937142 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 12:45:05.937296 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:05.937282 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 12:45:05.938066 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:05.938040 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 12:45:05.938199 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:05.938085 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wcv98\"" Jul 20 12:45:05.952792 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:05.952762 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:06.011025 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.010995 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:06.031803 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.031776 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb1dfcff-aa2f-477e-ab35-302441b374c3-crio-socket\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.031950 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.031816 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.031950 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.031869 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb1dfcff-aa2f-477e-ab35-302441b374c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.032030 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.031976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctw2x\" (UniqueName: \"kubernetes.io/projected/cb1dfcff-aa2f-477e-ab35-302441b374c3-kube-api-access-ctw2x\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.032030 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.032015 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb1dfcff-aa2f-477e-ab35-302441b374c3-data-volume\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.133267 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.133231 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb1dfcff-aa2f-477e-ab35-302441b374c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.133484 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.133414 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctw2x\" (UniqueName: \"kubernetes.io/projected/cb1dfcff-aa2f-477e-ab35-302441b374c3-kube-api-access-ctw2x\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.133484 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.133452 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb1dfcff-aa2f-477e-ab35-302441b374c3-data-volume\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.133604 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.133504 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb1dfcff-aa2f-477e-ab35-302441b374c3-crio-socket\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.133604 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.133541 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.133752 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:06.133680 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 12:45:06.133752 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.133714 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb1dfcff-aa2f-477e-ab35-302441b374c3-crio-socket\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.133843 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:06.133752 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls podName:cb1dfcff-aa2f-477e-ab35-302441b374c3 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:06.633721033 +0000 UTC m=+48.605926843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8gb2" (UID: "cb1dfcff-aa2f-477e-ab35-302441b374c3") : secret "insights-runtime-extractor-tls" not found Jul 20 12:45:06.133843 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.133786 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb1dfcff-aa2f-477e-ab35-302441b374c3-data-volume\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.133951 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.133923 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb1dfcff-aa2f-477e-ab35-302441b374c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.142292 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.142262 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctw2x\" (UniqueName: \"kubernetes.io/projected/cb1dfcff-aa2f-477e-ab35-302441b374c3-kube-api-access-ctw2x\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.305063 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.305017 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:06.406553 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.406511 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:06.505998 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.505943 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:06.605880 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.605798 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:06.638022 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.637980 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:06.638173 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:06.638139 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 12:45:06.638212 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:06.638200 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls podName:cb1dfcff-aa2f-477e-ab35-302441b374c3 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:07.638184319 +0000 UTC m=+49.610390139 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8gb2" (UID: "cb1dfcff-aa2f-477e-ab35-302441b374c3") : secret "insights-runtime-extractor-tls" not found Jul 20 12:45:06.770241 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.770202 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" event={"ID":"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca","Type":"ContainerStarted","Data":"1b3460ac134ea08d277ea96b4dea9e2bd8d168c434f3d8f12418e0088300c112"} Jul 20 12:45:06.771915 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.771895 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/0.log" Jul 20 12:45:06.772067 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.771930 2570 generic.go:356] "Generic (PLEG): container finished" podID="071ec3e8-c145-4cb7-9ab5-116975ec402c" containerID="e4a550adbe6f25971d97815148a9900b9f8b0f880290e2a5148089818262eb3c" exitCode=255 Jul 20 12:45:06.772067 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.772009 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" event={"ID":"071ec3e8-c145-4cb7-9ab5-116975ec402c","Type":"ContainerDied","Data":"e4a550adbe6f25971d97815148a9900b9f8b0f880290e2a5148089818262eb3c"} Jul 20 12:45:06.772243 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.772230 2570 scope.go:117] "RemoveContainer" containerID="e4a550adbe6f25971d97815148a9900b9f8b0f880290e2a5148089818262eb3c" Jul 20 12:45:06.773765 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.773738 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-rqn64" event={"ID":"72ab29c8-e484-4ffa-b85b-48848df8d63c","Type":"ContainerStarted","Data":"c816d697de49167842a4701448811ac6c9e201c77c5e34c81299fdd45e67cbc3"} Jul 20 12:45:06.775448 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.775421 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xkbwk" event={"ID":"6728fc86-fff5-4df8-8231-626b9b01e80d","Type":"ContainerStarted","Data":"1aa57f3831fa340de376a498c5a726655f65556f04a7b885f7a2ee8323d75d59"} Jul 20 12:45:06.787193 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.787130 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" podStartSLOduration=1.929806355 podStartE2EDuration="6.787110595s" totalImagesPullingTime="4.85730424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:06.78586321 +0000 UTC m=+48.758069079" watchObservedRunningTime="2026-07-20 12:45:06.787110595 +0000 UTC m=+48.759316425" Jul 20 12:45:06.819240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.819165 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-rqn64" podStartSLOduration=1.9937875539999999 podStartE2EDuration="6.819142619s" totalImagesPullingTime="4.825355065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:06.817762742 +0000 UTC m=+48.789968574" watchObservedRunningTime="2026-07-20 12:45:06.819142619 +0000 UTC m=+48.791348452" Jul 20 12:45:06.833395 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.831362 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xkbwk" podStartSLOduration=1.511232081 podStartE2EDuration="6.831341367s" totalImagesPullingTime="5.320109286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:06.83034487 +0000 UTC m=+48.802550702" watchObservedRunningTime="2026-07-20 12:45:06.831341367 +0000 UTC m=+48.803547199" Jul 20 12:45:06.901089 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:06.901054 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:07.002449 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.002419 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:07.377986 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.377951 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:07.409287 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.409255 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jstg9_c1fc1557-eab5-4747-9378-59a27b43a32a/dns-node-resolver/0.log" Jul 20 12:45:07.473949 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.473911 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:07.573814 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.573782 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:07.646689 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.646576 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:07.646866 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:07.646747 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 12:45:07.646866 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:07.646823 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls podName:cb1dfcff-aa2f-477e-ab35-302441b374c3 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:09.64680741 +0000 UTC m=+51.619013220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8gb2" (UID: "cb1dfcff-aa2f-477e-ab35-302441b374c3") : secret "insights-runtime-extractor-tls" not found Jul 20 12:45:07.780162 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.780138 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 12:45:07.780544 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.780528 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/0.log" Jul 20 12:45:07.780612 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.780564 2570 generic.go:356] "Generic (PLEG): container finished" podID="071ec3e8-c145-4cb7-9ab5-116975ec402c" containerID="46eb61e837bd6cc7aecd59b925c85f4f35d4cfea9ebc8636fff5b183344ade0f" exitCode=255 Jul 20 12:45:07.780692 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.780668 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" event={"ID":"071ec3e8-c145-4cb7-9ab5-116975ec402c","Type":"ContainerDied","Data":"46eb61e837bd6cc7aecd59b925c85f4f35d4cfea9ebc8636fff5b183344ade0f"} Jul 20 12:45:07.780744 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.780732 2570 scope.go:117] "RemoveContainer" containerID="e4a550adbe6f25971d97815148a9900b9f8b0f880290e2a5148089818262eb3c" Jul 20 12:45:07.780983 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:07.780964 2570 scope.go:117] "RemoveContainer" containerID="46eb61e837bd6cc7aecd59b925c85f4f35d4cfea9ebc8636fff5b183344ade0f" Jul 20 12:45:07.781181 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:07.781154 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-cwqb4_openshift-console-operator(071ec3e8-c145-4cb7-9ab5-116975ec402c)\"" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" podUID="071ec3e8-c145-4cb7-9ab5-116975ec402c" Jul 20 12:45:08.096560 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.096529 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-8zdr8"] Jul 20 12:45:08.110843 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.110814 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-8zdr8"] Jul 20 12:45:08.110978 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.110932 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-8zdr8" Jul 20 12:45:08.113030 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.113003 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-98zmc\"" Jul 20 12:45:08.113174 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.113054 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 20 12:45:08.113174 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.113099 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 20 12:45:08.113561 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.113531 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 20 12:45:08.113561 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.113554 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 20 12:45:08.130844 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.130795 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:08.151187 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.151152 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8d59d136-c194-4ad2-b37a-d067ca6b8c4f-signing-key\") pod \"service-ca-58598695f7-8zdr8\" (UID: \"8d59d136-c194-4ad2-b37a-d067ca6b8c4f\") " pod="openshift-service-ca/service-ca-58598695f7-8zdr8" Jul 20 12:45:08.151331 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.151200 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmx8d\" (UniqueName: \"kubernetes.io/projected/8d59d136-c194-4ad2-b37a-d067ca6b8c4f-kube-api-access-zmx8d\") pod \"service-ca-58598695f7-8zdr8\" (UID: \"8d59d136-c194-4ad2-b37a-d067ca6b8c4f\") " pod="openshift-service-ca/service-ca-58598695f7-8zdr8" Jul 20 12:45:08.151447 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.151325 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8d59d136-c194-4ad2-b37a-d067ca6b8c4f-signing-cabundle\") pod \"service-ca-58598695f7-8zdr8\" (UID: \"8d59d136-c194-4ad2-b37a-d067ca6b8c4f\") " pod="openshift-service-ca/service-ca-58598695f7-8zdr8" Jul 20 12:45:08.252432 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.252393 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8d59d136-c194-4ad2-b37a-d067ca6b8c4f-signing-key\") pod \"service-ca-58598695f7-8zdr8\" (UID: \"8d59d136-c194-4ad2-b37a-d067ca6b8c4f\") " pod="openshift-service-ca/service-ca-58598695f7-8zdr8" Jul 20 12:45:08.252432 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.252441 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmx8d\" (UniqueName: \"kubernetes.io/projected/8d59d136-c194-4ad2-b37a-d067ca6b8c4f-kube-api-access-zmx8d\") pod \"service-ca-58598695f7-8zdr8\" (UID: \"8d59d136-c194-4ad2-b37a-d067ca6b8c4f\") " pod="openshift-service-ca/service-ca-58598695f7-8zdr8" Jul 20 12:45:08.252631 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.252506 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8d59d136-c194-4ad2-b37a-d067ca6b8c4f-signing-cabundle\") pod \"service-ca-58598695f7-8zdr8\" (UID: \"8d59d136-c194-4ad2-b37a-d067ca6b8c4f\") " pod="openshift-service-ca/service-ca-58598695f7-8zdr8" Jul 20 12:45:08.253155 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.253136 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8d59d136-c194-4ad2-b37a-d067ca6b8c4f-signing-cabundle\") pod \"service-ca-58598695f7-8zdr8\" (UID: \"8d59d136-c194-4ad2-b37a-d067ca6b8c4f\") " pod="openshift-service-ca/service-ca-58598695f7-8zdr8" Jul 20 12:45:08.254798 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.254779 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8d59d136-c194-4ad2-b37a-d067ca6b8c4f-signing-key\") pod \"service-ca-58598695f7-8zdr8\" (UID: \"8d59d136-c194-4ad2-b37a-d067ca6b8c4f\") " pod="openshift-service-ca/service-ca-58598695f7-8zdr8" Jul 20 12:45:08.259566 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.259541 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmx8d\" (UniqueName: \"kubernetes.io/projected/8d59d136-c194-4ad2-b37a-d067ca6b8c4f-kube-api-access-zmx8d\") pod \"service-ca-58598695f7-8zdr8\" (UID: \"8d59d136-c194-4ad2-b37a-d067ca6b8c4f\") " pod="openshift-service-ca/service-ca-58598695f7-8zdr8" Jul 20 12:45:08.353898 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.353805 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:08.354075 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:08.353955 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 12:45:08.354075 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.353983 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:08.354075 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:08.354018 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:16.354003554 +0000 UTC m=+58.326209365 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : secret "router-metrics-certs-default" not found Jul 20 12:45:08.354226 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:08.354099 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:16.354083789 +0000 UTC m=+58.326289598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : configmap references non-existent config key: service-ca.crt Jul 20 12:45:08.420036 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.420008 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-8zdr8" Jul 20 12:45:08.454433 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.454403 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:08.454607 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:08.454589 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 12:45:08.454653 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:08.454609 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fcfdc44d-flp46: secret "image-registry-tls" not found Jul 20 12:45:08.454703 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:08.454673 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls podName:24c7d17a-2831-40f6-a1cf-a07bb02fc3e0 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:16.454655628 +0000 UTC m=+58.426861438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls") pod "image-registry-6fcfdc44d-flp46" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0") : secret "image-registry-tls" not found Jul 20 12:45:08.571147 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.571108 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-8zdr8"] Jul 20 12:45:08.575028 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:08.574991 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d59d136_c194_4ad2_b37a_d067ca6b8c4f.slice/crio-4caa11384da1f82fc826752200f964d6153805789c0b6d945c59e334f7888b5b WatchSource:0}: Error finding container 4caa11384da1f82fc826752200f964d6153805789c0b6d945c59e334f7888b5b: Status 404 returned error can't find the container with id 4caa11384da1f82fc826752200f964d6153805789c0b6d945c59e334f7888b5b Jul 20 12:45:08.784782 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.784707 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 12:45:08.785080 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.785062 2570 scope.go:117] "RemoveContainer" containerID="46eb61e837bd6cc7aecd59b925c85f4f35d4cfea9ebc8636fff5b183344ade0f" Jul 20 12:45:08.785243 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:08.785216 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-cwqb4_openshift-console-operator(071ec3e8-c145-4cb7-9ab5-116975ec402c)\"" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" podUID="071ec3e8-c145-4cb7-9ab5-116975ec402c" Jul 20 12:45:08.785956 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.785932 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-8zdr8" event={"ID":"8d59d136-c194-4ad2-b37a-d067ca6b8c4f","Type":"ContainerStarted","Data":"4caa11384da1f82fc826752200f964d6153805789c0b6d945c59e334f7888b5b"} Jul 20 12:45:08.813797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:08.813771 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vs9qv_fd720144-9924-46ef-a880-0c196604451b/node-ca/0.log" Jul 20 12:45:09.070657 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:09.070572 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:09.202831 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:09.202795 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8zhcc_f09b4592-c85d-4f65-b6d1-63e0d175afa4/serve-healthcheck-canary/0.log" Jul 20 12:45:09.261242 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:09.261198 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:45:09.261452 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:09.261332 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 12:45:09.261525 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:09.261495 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls podName:8319b26b-2857-4f07-aeae-9116dbb18e10 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:25.261403254 +0000 UTC m=+67.233609083 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls") pod "dns-default-tlzx5" (UID: "8319b26b-2857-4f07-aeae-9116dbb18e10") : secret "dns-default-metrics-tls" not found Jul 20 12:45:09.666136 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:09.666095 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:09.666324 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:09.666223 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 12:45:09.666324 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:09.666290 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls podName:cb1dfcff-aa2f-477e-ab35-302441b374c3 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:13.666270634 +0000 UTC m=+55.638476445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8gb2" (UID: "cb1dfcff-aa2f-477e-ab35-302441b374c3") : secret "insights-runtime-extractor-tls" not found Jul 20 12:45:10.828405 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:10.828355 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:10.828786 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:10.828474 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:10.828786 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:10.828733 2570 scope.go:117] "RemoveContainer" containerID="46eb61e837bd6cc7aecd59b925c85f4f35d4cfea9ebc8636fff5b183344ade0f" Jul 20 12:45:10.828892 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:10.828874 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-cwqb4_openshift-console-operator(071ec3e8-c145-4cb7-9ab5-116975ec402c)\"" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" podUID="071ec3e8-c145-4cb7-9ab5-116975ec402c" Jul 20 12:45:11.788531 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:11.788493 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:11.796873 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:11.796840 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-8zdr8" event={"ID":"8d59d136-c194-4ad2-b37a-d067ca6b8c4f","Type":"ContainerStarted","Data":"1e68d581f7184a419ba76a1d3a2b6fe8454e3042f7064e2fa9c2baa34efbef25"} Jul 20 12:45:11.797129 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:11.797115 2570 scope.go:117] "RemoveContainer" containerID="46eb61e837bd6cc7aecd59b925c85f4f35d4cfea9ebc8636fff5b183344ade0f" Jul 20 12:45:11.797272 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:11.797254 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-cwqb4_openshift-console-operator(071ec3e8-c145-4cb7-9ab5-116975ec402c)\"" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" podUID="071ec3e8-c145-4cb7-9ab5-116975ec402c" Jul 20 12:45:11.812014 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:11.811957 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-8zdr8" podStartSLOduration=1.5206334080000001 podStartE2EDuration="3.811939345s" totalImagesPullingTime="2.291305937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:11.810438175 +0000 UTC m=+53.782644007" watchObservedRunningTime="2026-07-20 12:45:11.811939345 +0000 UTC m=+53.784145179" Jul 20 12:45:12.215729 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:12.215691 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:12.297492 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:12.297451 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:12.397492 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:12.397458 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:12.497046 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:12.496941 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:13.382276 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:13.382238 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:13.580510 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:13.580476 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:13.681001 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:13.680912 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:13.703208 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:13.703164 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:13.703341 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:13.703320 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 12:45:13.703431 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:13.703419 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls podName:cb1dfcff-aa2f-477e-ab35-302441b374c3 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:21.703398627 +0000 UTC m=+63.675604440 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8gb2" (UID: "cb1dfcff-aa2f-477e-ab35-302441b374c3") : secret "insights-runtime-extractor-tls" not found Jul 20 12:45:13.782507 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:13.782469 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:13.982299 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:13.982213 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:14.179407 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:14.179358 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:14.779506 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:14.779471 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:14.980278 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:14.980246 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:15.249197 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:15.249166 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:15.795973 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:15.795936 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:16.076067 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.075986 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:16.181631 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.181596 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:16.390532 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.390480 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:16.426875 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.426835 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:16.427037 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:16.427016 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:32.426995546 +0000 UTC m=+74.399201378 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : configmap references non-existent config key: service-ca.crt Jul 20 12:45:16.427116 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.427101 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:16.427249 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:16.427228 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 12:45:16.427302 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:16.427295 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs podName:e66b1363-7be7-4cad-ad86-053fbe8c5596 nodeName:}" failed. No retries permitted until 2026-07-20 12:45:32.427277454 +0000 UTC m=+74.399483270 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs") pod "router-default-594bbb7854-524rq" (UID: "e66b1363-7be7-4cad-ad86-053fbe8c5596") : secret "router-metrics-certs-default" not found Jul 20 12:45:16.527733 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.527689 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:16.530295 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.530264 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls\") pod \"image-registry-6fcfdc44d-flp46\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:16.542163 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.542123 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:16.677999 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.677896 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fcfdc44d-flp46"] Jul 20 12:45:16.681618 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:16.681582 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c7d17a_2831_40f6_a1cf_a07bb02fc3e0.slice/crio-fea8a7bfe203f1bf354c071d5ecb4c4384126b129a29ab3fa799206ae04d8113 WatchSource:0}: Error finding container fea8a7bfe203f1bf354c071d5ecb4c4384126b129a29ab3fa799206ae04d8113: Status 404 returned error can't find the container with id fea8a7bfe203f1bf354c071d5ecb4c4384126b129a29ab3fa799206ae04d8113 Jul 20 12:45:16.779481 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.779446 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:16.815535 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.815500 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" event={"ID":"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0","Type":"ContainerStarted","Data":"243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808"} Jul 20 12:45:16.815948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.815554 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" event={"ID":"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0","Type":"ContainerStarted","Data":"fea8a7bfe203f1bf354c071d5ecb4c4384126b129a29ab3fa799206ae04d8113"} Jul 20 12:45:16.815948 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.815657 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:16.837851 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.837793 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" podStartSLOduration=16.837778161 podStartE2EDuration="16.837778161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:16.837269659 +0000 UTC m=+58.809475493" watchObservedRunningTime="2026-07-20 12:45:16.837778161 +0000 UTC m=+58.809983993" Jul 20 12:45:16.983782 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:16.983698 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:17.041833 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:17.041802 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:17.379434 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:17.379402 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:17.984565 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:17.984531 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:18.578333 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:18.578303 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:18.777642 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:18.777606 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:19.182929 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:19.182894 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:19.283927 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:19.283896 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:19.383638 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:19.383608 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:19.716863 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:19.716834 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z95dr" Jul 20 12:45:19.777734 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:19.777696 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:20.164053 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:20.164022 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:20.587592 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:20.587558 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:20.779479 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:20.779449 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:20.984236 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:20.984150 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:21.389359 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:21.389328 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:21.769954 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:21.769858 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:21.772227 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:21.772194 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb1dfcff-aa2f-477e-ab35-302441b374c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8gb2\" (UID: \"cb1dfcff-aa2f-477e-ab35-302441b374c3\") " pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:21.847064 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:21.847027 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wcv98\"" Jul 20 12:45:21.855883 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:21.855842 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m8gb2" Jul 20 12:45:21.978654 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:21.978627 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:21.983798 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:21.983769 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m8gb2"] Jul 20 12:45:21.986850 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:21.986818 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb1dfcff_aa2f_477e_ab35_302441b374c3.slice/crio-fc4f73f9956dcd89a7ed30e1a2efc140649ecb331eddb6897d05ba38ef61c7ef WatchSource:0}: Error finding container fc4f73f9956dcd89a7ed30e1a2efc140649ecb331eddb6897d05ba38ef61c7ef: Status 404 returned error can't find the container with id fc4f73f9956dcd89a7ed30e1a2efc140649ecb331eddb6897d05ba38ef61c7ef Jul 20 12:45:22.377409 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:22.377355 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:22.516240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:22.516205 2570 scope.go:117] "RemoveContainer" containerID="46eb61e837bd6cc7aecd59b925c85f4f35d4cfea9ebc8636fff5b183344ade0f" Jul 20 12:45:22.580383 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:22.580337 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:22.775474 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:22.775118 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:22.838579 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:22.838549 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 12:45:22.838776 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:22.838652 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" event={"ID":"071ec3e8-c145-4cb7-9ab5-116975ec402c","Type":"ContainerStarted","Data":"efc85e67587d837ecdafebc6307b87787c1dad061a1ab89e0b3a9a9ff63bfc68"} Jul 20 12:45:22.839332 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:22.839311 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:22.842703 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:22.842671 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8gb2" event={"ID":"cb1dfcff-aa2f-477e-ab35-302441b374c3","Type":"ContainerStarted","Data":"fc4f73f9956dcd89a7ed30e1a2efc140649ecb331eddb6897d05ba38ef61c7ef"} Jul 20 12:45:22.855309 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:22.855250 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" podStartSLOduration=17.940091353 podStartE2EDuration="22.85522818s" totalImagesPullingTime="4.915136827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:22.854265323 +0000 UTC m=+64.826471157" watchObservedRunningTime="2026-07-20 12:45:22.85522818 +0000 UTC m=+64.827434012" Jul 20 12:45:23.508053 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:23.508025 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-cwqb4" Jul 20 12:45:23.698605 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:23.698561 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:23.816194 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:23.816104 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:23.901202 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:23.901171 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:24.000807 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.000773 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:24.161703 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.161667 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:24.222486 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.222444 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:24.295548 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.295502 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:45:24.295739 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.295580 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:45:24.297814 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.297790 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 12:45:24.298266 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.298239 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 12:45:24.308225 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.308199 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1220141b-d9f7-4e87-95e8-cc658b486b7f-original-pull-secret\") pod \"global-pull-secret-syncer-wvc67\" (UID: \"1220141b-d9f7-4e87-95e8-cc658b486b7f\") " pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:45:24.308402 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.308286 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24ad9090-71b2-4f16-9389-19f115738cf6-metrics-certs\") pod \"network-metrics-daemon-6gkrz\" (UID: \"24ad9090-71b2-4f16-9389-19f115738cf6\") " pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:45:24.396900 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.396866 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbvgj\" (UniqueName: \"kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj\") pod \"network-check-target-67vbc\" (UID: \"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72\") " pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:45:24.398839 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.398808 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:24.399549 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.399526 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbvgj\" (UniqueName: \"kubernetes.io/projected/6e808ebf-fa3a-423a-a77f-5ae8d13c6e72-kube-api-access-nbvgj\") pod \"network-check-target-67vbc\" (UID: \"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72\") " pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:45:24.428522 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.428442 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pvfjj\"" Jul 20 12:45:24.435065 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.435039 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cg6n8\"" Jul 20 12:45:24.437142 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.437111 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gkrz" Jul 20 12:45:24.439624 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.439590 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvc67" Jul 20 12:45:24.444458 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.444430 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:45:24.502073 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.502038 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:24.772797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:24.772714 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:25.305492 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:25.305451 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:45:25.307962 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:25.307932 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8319b26b-2857-4f07-aeae-9116dbb18e10-metrics-tls\") pod \"dns-default-tlzx5\" (UID: \"8319b26b-2857-4f07-aeae-9116dbb18e10\") " pod="openshift-dns/dns-default-tlzx5" Jul 20 12:45:25.379179 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:25.379144 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:25.477430 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:25.477403 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dcf82\"" Jul 20 12:45:25.485995 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:25.485964 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tlzx5" Jul 20 12:45:25.578696 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:25.578611 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:25.845192 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:25.845158 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:26.203787 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:26.203694 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:27.806737 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:27.806703 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6gkrz"] Jul 20 12:45:27.809480 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:27.809449 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ad9090_71b2_4f16_9389_19f115738cf6.slice/crio-7fe9c562f0f5faee0887c9cf0a1cf61cffd3d072dd37479f7a29b89618145024 WatchSource:0}: Error finding container 7fe9c562f0f5faee0887c9cf0a1cf61cffd3d072dd37479f7a29b89618145024: Status 404 returned error can't find the container with id 7fe9c562f0f5faee0887c9cf0a1cf61cffd3d072dd37479f7a29b89618145024 Jul 20 12:45:27.849194 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:27.849162 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-67vbc"] Jul 20 12:45:27.863724 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:27.863693 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6gkrz" event={"ID":"24ad9090-71b2-4f16-9389-19f115738cf6","Type":"ContainerStarted","Data":"7fe9c562f0f5faee0887c9cf0a1cf61cffd3d072dd37479f7a29b89618145024"} Jul 20 12:45:27.994733 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:27.994707 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e808ebf_fa3a_423a_a77f_5ae8d13c6e72.slice/crio-7957078894d324db3cf196805a29920ceb1af940184f2925ef639795a585c133 WatchSource:0}: Error finding container 7957078894d324db3cf196805a29920ceb1af940184f2925ef639795a585c133: Status 404 returned error can't find the container with id 7957078894d324db3cf196805a29920ceb1af940184f2925ef639795a585c133 Jul 20 12:45:28.027387 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.027345 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wvc67"] Jul 20 12:45:28.030189 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.030159 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tlzx5"] Jul 20 12:45:28.031105 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:28.031081 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1220141b_d9f7_4e87_95e8_cc658b486b7f.slice/crio-742c20987e5eab3ed67240163f271d5aab5b32c8b421e5b26d0042c2d5095a61 WatchSource:0}: Error finding container 742c20987e5eab3ed67240163f271d5aab5b32c8b421e5b26d0042c2d5095a61: Status 404 returned error can't find the container with id 742c20987e5eab3ed67240163f271d5aab5b32c8b421e5b26d0042c2d5095a61 Jul 20 12:45:28.033623 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:28.033599 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8319b26b_2857_4f07_aeae_9116dbb18e10.slice/crio-061b37430b1466cdd114a2e9b1cb02b0a46562c2a8867500eb7d6c1167c3612f WatchSource:0}: Error finding container 061b37430b1466cdd114a2e9b1cb02b0a46562c2a8867500eb7d6c1167c3612f: Status 404 returned error can't find the container with id 061b37430b1466cdd114a2e9b1cb02b0a46562c2a8867500eb7d6c1167c3612f Jul 20 12:45:28.259671 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.259581 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:28.367461 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.367430 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:28.501940 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.501901 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw"] Jul 20 12:45:28.522527 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.522301 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw" Jul 20 12:45:28.525284 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.525004 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 20 12:45:28.525284 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.525264 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ssb2n\"" Jul 20 12:45:28.525819 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.525573 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 20 12:45:28.529627 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.529600 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5"] Jul 20 12:45:28.542170 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.542082 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw"] Jul 20 12:45:28.542170 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.542131 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5"] Jul 20 12:45:28.542453 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.542233 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5" Jul 20 12:45:28.545431 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.545394 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 20 12:45:28.550682 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.550280 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 20 12:45:28.550682 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.550280 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-stqwx\"" Jul 20 12:45:28.550682 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.550280 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 20 12:45:28.551384 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.551005 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 20 12:45:28.590591 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.590216 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fcfdc44d-flp46"] Jul 20 12:45:28.629296 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.629219 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:28.632925 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.632890 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2v9m\" (UniqueName: \"kubernetes.io/projected/2390e0a0-e67e-409d-b69b-18b7ca98a47e-kube-api-access-l2v9m\") pod \"managed-serviceaccount-addon-agent-5f878857d6-xk5m5\" (UID: \"2390e0a0-e67e-409d-b69b-18b7ca98a47e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5" Jul 20 12:45:28.633110 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.632937 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a7ce0ee4-eaab-4666-87fa-4cb351d17e56-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xfvcw\" (UID: \"a7ce0ee4-eaab-4666-87fa-4cb351d17e56\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw" Jul 20 12:45:28.633110 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.632976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2390e0a0-e67e-409d-b69b-18b7ca98a47e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f878857d6-xk5m5\" (UID: \"2390e0a0-e67e-409d-b69b-18b7ca98a47e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5" Jul 20 12:45:28.633110 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.633032 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a7ce0ee4-eaab-4666-87fa-4cb351d17e56-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xfvcw\" (UID: \"a7ce0ee4-eaab-4666-87fa-4cb351d17e56\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw" Jul 20 12:45:28.638588 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.638556 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-hpsh8"] Jul 20 12:45:28.653562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.653526 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-hpsh8" Jul 20 12:45:28.658133 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.657824 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 12:45:28.661172 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.658843 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 12:45:28.661172 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.660938 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-tg25v\"" Jul 20 12:45:28.668455 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.668378 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-hpsh8"] Jul 20 12:45:28.734351 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.734210 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj7m2\" (UniqueName: \"kubernetes.io/projected/615ac5be-f68e-4381-a35a-5ce6ac9df15b-kube-api-access-hj7m2\") pod \"downloads-5bc67dcd46-hpsh8\" (UID: \"615ac5be-f68e-4381-a35a-5ce6ac9df15b\") " pod="openshift-console/downloads-5bc67dcd46-hpsh8" Jul 20 12:45:28.734561 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.734356 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2v9m\" (UniqueName: \"kubernetes.io/projected/2390e0a0-e67e-409d-b69b-18b7ca98a47e-kube-api-access-l2v9m\") pod \"managed-serviceaccount-addon-agent-5f878857d6-xk5m5\" (UID: \"2390e0a0-e67e-409d-b69b-18b7ca98a47e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5" Jul 20 12:45:28.734561 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.734485 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a7ce0ee4-eaab-4666-87fa-4cb351d17e56-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xfvcw\" (UID: \"a7ce0ee4-eaab-4666-87fa-4cb351d17e56\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw" Jul 20 12:45:28.734561 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.734538 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2390e0a0-e67e-409d-b69b-18b7ca98a47e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f878857d6-xk5m5\" (UID: \"2390e0a0-e67e-409d-b69b-18b7ca98a47e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5" Jul 20 12:45:28.734729 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.734604 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a7ce0ee4-eaab-4666-87fa-4cb351d17e56-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xfvcw\" (UID: \"a7ce0ee4-eaab-4666-87fa-4cb351d17e56\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw" Jul 20 12:45:28.735785 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.735759 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a7ce0ee4-eaab-4666-87fa-4cb351d17e56-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xfvcw\" (UID: \"a7ce0ee4-eaab-4666-87fa-4cb351d17e56\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw" Jul 20 12:45:28.738298 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.738219 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a7ce0ee4-eaab-4666-87fa-4cb351d17e56-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xfvcw\" (UID: \"a7ce0ee4-eaab-4666-87fa-4cb351d17e56\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw" Jul 20 12:45:28.740390 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.739651 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2390e0a0-e67e-409d-b69b-18b7ca98a47e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f878857d6-xk5m5\" (UID: \"2390e0a0-e67e-409d-b69b-18b7ca98a47e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5" Jul 20 12:45:28.740390 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.739655 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8f5ccc466-5ksdn"] Jul 20 12:45:28.750436 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.750406 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.754104 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.754068 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2v9m\" (UniqueName: \"kubernetes.io/projected/2390e0a0-e67e-409d-b69b-18b7ca98a47e-kube-api-access-l2v9m\") pod \"managed-serviceaccount-addon-agent-5f878857d6-xk5m5\" (UID: \"2390e0a0-e67e-409d-b69b-18b7ca98a47e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5" Jul 20 12:45:28.761435 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.761381 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8f5ccc466-5ksdn"] Jul 20 12:45:28.793970 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.793936 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:28.835893 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.835854 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2tlc\" (UniqueName: \"kubernetes.io/projected/b23c4f77-5615-4e72-ad77-2f228693db38-kube-api-access-x2tlc\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.836397 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.835933 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b23c4f77-5615-4e72-ad77-2f228693db38-trusted-ca\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.836397 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.835965 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b23c4f77-5615-4e72-ad77-2f228693db38-image-registry-private-configuration\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.836397 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.836012 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj7m2\" (UniqueName: \"kubernetes.io/projected/615ac5be-f68e-4381-a35a-5ce6ac9df15b-kube-api-access-hj7m2\") pod \"downloads-5bc67dcd46-hpsh8\" (UID: \"615ac5be-f68e-4381-a35a-5ce6ac9df15b\") " pod="openshift-console/downloads-5bc67dcd46-hpsh8" Jul 20 12:45:28.836397 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.836045 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b23c4f77-5615-4e72-ad77-2f228693db38-ca-trust-extracted\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.836397 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.836072 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b23c4f77-5615-4e72-ad77-2f228693db38-bound-sa-token\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.836397 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.836130 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b23c4f77-5615-4e72-ad77-2f228693db38-registry-tls\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.836397 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.836170 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b23c4f77-5615-4e72-ad77-2f228693db38-installation-pull-secrets\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.836397 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.836212 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b23c4f77-5615-4e72-ad77-2f228693db38-registry-certificates\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.836755 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.836521 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw" Jul 20 12:45:28.855309 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.855238 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj7m2\" (UniqueName: \"kubernetes.io/projected/615ac5be-f68e-4381-a35a-5ce6ac9df15b-kube-api-access-hj7m2\") pod \"downloads-5bc67dcd46-hpsh8\" (UID: \"615ac5be-f68e-4381-a35a-5ce6ac9df15b\") " pod="openshift-console/downloads-5bc67dcd46-hpsh8" Jul 20 12:45:28.862639 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.862602 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5" Jul 20 12:45:28.870515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.870417 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8gb2" event={"ID":"cb1dfcff-aa2f-477e-ab35-302441b374c3","Type":"ContainerStarted","Data":"3adc90b34f3b363d13c8ec9bdc97c746efb7b810154b316bb3d5f37b45c60d14"} Jul 20 12:45:28.872491 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.872414 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wvc67" event={"ID":"1220141b-d9f7-4e87-95e8-cc658b486b7f","Type":"ContainerStarted","Data":"742c20987e5eab3ed67240163f271d5aab5b32c8b421e5b26d0042c2d5095a61"} Jul 20 12:45:28.876529 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.876491 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-67vbc" event={"ID":"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72","Type":"ContainerStarted","Data":"6a21c9b37672785b0518f3fe25507b4bb9102f0d7c660e865dff9368c29fa877"} Jul 20 12:45:28.876710 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.876555 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-67vbc" event={"ID":"6e808ebf-fa3a-423a-a77f-5ae8d13c6e72","Type":"ContainerStarted","Data":"7957078894d324db3cf196805a29920ceb1af940184f2925ef639795a585c133"} Jul 20 12:45:28.877294 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.877271 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:45:28.881576 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.881515 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tlzx5" event={"ID":"8319b26b-2857-4f07-aeae-9116dbb18e10","Type":"ContainerStarted","Data":"061b37430b1466cdd114a2e9b1cb02b0a46562c2a8867500eb7d6c1167c3612f"} Jul 20 12:45:28.899244 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.897809 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-67vbc" podStartSLOduration=69.89778779 podStartE2EDuration="1m9.89778779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:28.897524152 +0000 UTC m=+70.869729985" watchObservedRunningTime="2026-07-20 12:45:28.89778779 +0000 UTC m=+70.869993619" Jul 20 12:45:28.908635 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.908597 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:28.936892 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.936849 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2tlc\" (UniqueName: \"kubernetes.io/projected/b23c4f77-5615-4e72-ad77-2f228693db38-kube-api-access-x2tlc\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.937082 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.936934 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b23c4f77-5615-4e72-ad77-2f228693db38-trusted-ca\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.937082 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.936966 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b23c4f77-5615-4e72-ad77-2f228693db38-image-registry-private-configuration\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.937082 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.937021 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b23c4f77-5615-4e72-ad77-2f228693db38-ca-trust-extracted\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.937082 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.937050 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b23c4f77-5615-4e72-ad77-2f228693db38-bound-sa-token\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.937291 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.937105 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b23c4f77-5615-4e72-ad77-2f228693db38-registry-tls\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.937291 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.937140 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b23c4f77-5615-4e72-ad77-2f228693db38-installation-pull-secrets\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.937291 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.937182 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b23c4f77-5615-4e72-ad77-2f228693db38-registry-certificates\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.938194 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.938166 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b23c4f77-5615-4e72-ad77-2f228693db38-ca-trust-extracted\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.939361 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.939336 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b23c4f77-5615-4e72-ad77-2f228693db38-trusted-ca\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.939954 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.939931 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b23c4f77-5615-4e72-ad77-2f228693db38-registry-certificates\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.945747 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.945594 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b23c4f77-5615-4e72-ad77-2f228693db38-image-registry-private-configuration\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.947655 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.947624 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b23c4f77-5615-4e72-ad77-2f228693db38-installation-pull-secrets\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.949583 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.949555 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b23c4f77-5615-4e72-ad77-2f228693db38-registry-tls\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.959316 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.959279 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2tlc\" (UniqueName: \"kubernetes.io/projected/b23c4f77-5615-4e72-ad77-2f228693db38-kube-api-access-x2tlc\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.960543 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.960514 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b23c4f77-5615-4e72-ad77-2f228693db38-bound-sa-token\") pod \"image-registry-8f5ccc466-5ksdn\" (UID: \"b23c4f77-5615-4e72-ad77-2f228693db38\") " pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:28.980817 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:28.980780 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-hpsh8" Jul 20 12:45:29.000731 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.000689 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:29.079860 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.079779 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:29.493024 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.492944 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5"] Jul 20 12:45:29.501997 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:29.501963 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2390e0a0_e67e_409d_b69b_18b7ca98a47e.slice/crio-a00023a13f2379b2e9e8f10a40f3cb5cf6bb385600d9f36beaa11eaa53df9709 WatchSource:0}: Error finding container a00023a13f2379b2e9e8f10a40f3cb5cf6bb385600d9f36beaa11eaa53df9709: Status 404 returned error can't find the container with id a00023a13f2379b2e9e8f10a40f3cb5cf6bb385600d9f36beaa11eaa53df9709 Jul 20 12:45:29.730533 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.730435 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-hpsh8"] Jul 20 12:45:29.736486 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.736450 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw"] Jul 20 12:45:29.736599 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.736528 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8f5ccc466-5ksdn"] Jul 20 12:45:29.739078 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:29.739044 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615ac5be_f68e_4381_a35a_5ce6ac9df15b.slice/crio-8a2b7776af94420efa62abd698c0fed79ab736d32138b49fb22deeb66a7af9a4 WatchSource:0}: Error finding container 8a2b7776af94420efa62abd698c0fed79ab736d32138b49fb22deeb66a7af9a4: Status 404 returned error can't find the container with id 8a2b7776af94420efa62abd698c0fed79ab736d32138b49fb22deeb66a7af9a4 Jul 20 12:45:29.744088 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:29.744059 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23c4f77_5615_4e72_ad77_2f228693db38.slice/crio-8906d114c8597c4ce3fca76787582c6d6bc67162d946b051bf70bffde4cd29d0 WatchSource:0}: Error finding container 8906d114c8597c4ce3fca76787582c6d6bc67162d946b051bf70bffde4cd29d0: Status 404 returned error can't find the container with id 8906d114c8597c4ce3fca76787582c6d6bc67162d946b051bf70bffde4cd29d0 Jul 20 12:45:29.745074 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:29.745046 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ce0ee4_eaab_4666_87fa_4cb351d17e56.slice/crio-4e119ef3ee1501dfc8db82c547ba0b089fe91069417d7f5e5d5b095f6c5644cb WatchSource:0}: Error finding container 4e119ef3ee1501dfc8db82c547ba0b089fe91069417d7f5e5d5b095f6c5644cb: Status 404 returned error can't find the container with id 4e119ef3ee1501dfc8db82c547ba0b089fe91069417d7f5e5d5b095f6c5644cb Jul 20 12:45:29.887832 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.887744 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw" event={"ID":"a7ce0ee4-eaab-4666-87fa-4cb351d17e56","Type":"ContainerStarted","Data":"4e119ef3ee1501dfc8db82c547ba0b089fe91069417d7f5e5d5b095f6c5644cb"} Jul 20 12:45:29.890191 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.890156 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-hpsh8" event={"ID":"615ac5be-f68e-4381-a35a-5ce6ac9df15b","Type":"ContainerStarted","Data":"8a2b7776af94420efa62abd698c0fed79ab736d32138b49fb22deeb66a7af9a4"} Jul 20 12:45:29.892544 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.892507 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" event={"ID":"b23c4f77-5615-4e72-ad77-2f228693db38","Type":"ContainerStarted","Data":"3e8c59871dc74102c4bd7330f267adf0a7117878cdb675d3456f78f24f4ceedc"} Jul 20 12:45:29.892700 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.892572 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" event={"ID":"b23c4f77-5615-4e72-ad77-2f228693db38","Type":"ContainerStarted","Data":"8906d114c8597c4ce3fca76787582c6d6bc67162d946b051bf70bffde4cd29d0"} Jul 20 12:45:29.892700 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.892614 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:29.894529 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.894498 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5" event={"ID":"2390e0a0-e67e-409d-b69b-18b7ca98a47e","Type":"ContainerStarted","Data":"a00023a13f2379b2e9e8f10a40f3cb5cf6bb385600d9f36beaa11eaa53df9709"} Jul 20 12:45:29.898984 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.898952 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6gkrz" event={"ID":"24ad9090-71b2-4f16-9389-19f115738cf6","Type":"ContainerStarted","Data":"a915d6b7c0ae1fbe7519c9284e7e04f93457f3fa85cd4db19bb7545503005a67"} Jul 20 12:45:29.899130 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.899007 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6gkrz" event={"ID":"24ad9090-71b2-4f16-9389-19f115738cf6","Type":"ContainerStarted","Data":"9f862dadec12c92d5db7617c7f3d4464127b7f3d7ce182153d23aad06e25cb29"} Jul 20 12:45:29.911028 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.910968 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" podStartSLOduration=1.910946601 podStartE2EDuration="1.910946601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:29.90987736 +0000 UTC m=+71.882083194" watchObservedRunningTime="2026-07-20 12:45:29.910946601 +0000 UTC m=+71.883152431" Jul 20 12:45:29.924717 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:29.924655 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6gkrz" podStartSLOduration=69.648407673 podStartE2EDuration="1m10.924627228s" totalImagesPullingTime="1.276219555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:29.923783507 +0000 UTC m=+71.895989340" watchObservedRunningTime="2026-07-20 12:45:29.924627228 +0000 UTC m=+71.896833063" Jul 20 12:45:30.048584 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:30.048549 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:30.153483 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:30.153319 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:31.224898 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:31.224862 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:31.325719 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:31.325660 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:32.470479 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:32.470438 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:32.470968 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:32.470530 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:32.471286 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:32.471233 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e66b1363-7be7-4cad-ad86-053fbe8c5596-service-ca-bundle\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:32.473271 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:32.473249 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e66b1363-7be7-4cad-ad86-053fbe8c5596-metrics-certs\") pod \"router-default-594bbb7854-524rq\" (UID: \"e66b1363-7be7-4cad-ad86-053fbe8c5596\") " pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:32.656975 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:32.656938 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jkxmq\"" Jul 20 12:45:32.665574 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:32.665538 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:34.219952 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:34.219910 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:34.273411 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:34.273347 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:34.325480 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:34.325444 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-594bbb7854-524rq"] Jul 20 12:45:36.172819 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:36.172775 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode66b1363_7be7_4cad_ad86_053fbe8c5596.slice/crio-123ddc0a95744eac07cc56ee95512735ae4ce992963e4715ff36e6fe1363df57 WatchSource:0}: Error finding container 123ddc0a95744eac07cc56ee95512735ae4ce992963e4715ff36e6fe1363df57: Status 404 returned error can't find the container with id 123ddc0a95744eac07cc56ee95512735ae4ce992963e4715ff36e6fe1363df57 Jul 20 12:45:36.604079 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.604021 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:36.706155 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.706120 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:36.934391 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.934334 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5" event={"ID":"2390e0a0-e67e-409d-b69b-18b7ca98a47e","Type":"ContainerStarted","Data":"d03d7e81f9016b716cddf847d749981aada4c566685e26ba641e624be19f0db8"} Jul 20 12:45:36.937885 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.937780 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tlzx5" event={"ID":"8319b26b-2857-4f07-aeae-9116dbb18e10","Type":"ContainerStarted","Data":"132938a786e6f3a49a7ba3cd113ea2d29665ff62daa3510895b97b8b873f1040"} Jul 20 12:45:36.941620 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.941590 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw" event={"ID":"a7ce0ee4-eaab-4666-87fa-4cb351d17e56","Type":"ContainerStarted","Data":"96ac9acf67183e5a60d984474dfb0dcac96700832a20904f0e88637332e902e1"} Jul 20 12:45:36.944803 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.944768 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8gb2" event={"ID":"cb1dfcff-aa2f-477e-ab35-302441b374c3","Type":"ContainerStarted","Data":"6830927197e8b86e078d4ec8963b0635ab578eac7aba8165cb67109abbc30562"} Jul 20 12:45:36.948577 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.948539 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wvc67" event={"ID":"1220141b-d9f7-4e87-95e8-cc658b486b7f","Type":"ContainerStarted","Data":"1a96c29391f33d47aa3da6bc7d576bc6eccf47267ab65432581d739b6ca53ba4"} Jul 20 12:45:36.950737 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.950649 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f878857d6-xk5m5" podStartSLOduration=1.8027896700000001 podStartE2EDuration="8.950626899s" totalImagesPullingTime="7.147837229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:36.95030202 +0000 UTC m=+78.922507853" watchObservedRunningTime="2026-07-20 12:45:36.950626899 +0000 UTC m=+78.922832735" Jul 20 12:45:36.952285 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.952259 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-594bbb7854-524rq" event={"ID":"e66b1363-7be7-4cad-ad86-053fbe8c5596","Type":"ContainerStarted","Data":"7aff0a56a2f4171ced8c5fdd96e857c9fc9375f9734fa1643a975e0a1e68faf1"} Jul 20 12:45:36.952443 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.952302 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-594bbb7854-524rq" event={"ID":"e66b1363-7be7-4cad-ad86-053fbe8c5596","Type":"ContainerStarted","Data":"123ddc0a95744eac07cc56ee95512735ae4ce992963e4715ff36e6fe1363df57"} Jul 20 12:45:36.968876 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.968818 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xfvcw" podStartSLOduration=2.060431728 podStartE2EDuration="8.968796102s" totalImagesPullingTime="6.908364374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:36.968444507 +0000 UTC m=+78.940650351" watchObservedRunningTime="2026-07-20 12:45:36.968796102 +0000 UTC m=+78.941001934" Jul 20 12:45:36.996810 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:36.995916 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wvc67" podStartSLOduration=69.420759829 podStartE2EDuration="1m17.995894664s" totalImagesPullingTime="8.575134835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:36.995465724 +0000 UTC m=+78.967671557" watchObservedRunningTime="2026-07-20 12:45:36.995894664 +0000 UTC m=+78.968100497" Jul 20 12:45:37.016779 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:37.016469 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-594bbb7854-524rq" podStartSLOduration=37.016449423 podStartE2EDuration="37.016449423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:37.01504537 +0000 UTC m=+78.987251205" watchObservedRunningTime="2026-07-20 12:45:37.016449423 +0000 UTC m=+78.988655255" Jul 20 12:45:37.110055 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:37.109954 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:37.209888 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:37.209849 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:37.665971 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:37.665930 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:37.669171 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:37.669040 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:37.962487 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:37.962244 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tlzx5" event={"ID":"8319b26b-2857-4f07-aeae-9116dbb18e10","Type":"ContainerStarted","Data":"07302378493a46f28429d3e53b8a9e22913eff0182a28b31bc6f8145423588ba"} Jul 20 12:45:37.963138 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:37.963110 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tlzx5" Jul 20 12:45:37.963402 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:37.963156 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:37.964456 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:37.964434 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-594bbb7854-524rq" Jul 20 12:45:37.981955 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:37.981899 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tlzx5" podStartSLOduration=38.831978679 podStartE2EDuration="44.981879892s" totalImagesPullingTime="6.149901213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:37.980642777 +0000 UTC m=+79.952848609" watchObservedRunningTime="2026-07-20 12:45:37.981879892 +0000 UTC m=+79.954085752" Jul 20 12:45:38.597550 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:38.597503 2570 patch_prober.go:28] interesting pod/image-registry-6fcfdc44d-flp46 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 12:45:38.597996 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:38.597563 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" podUID="24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 12:45:38.615488 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:38.615424 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:38.674645 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:38.674594 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:40.008698 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:40.008651 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:40.106400 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:40.106316 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:41.649216 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:41.649180 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:41.750908 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:41.750868 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:41.822173 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:41.822138 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:41.986582 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:41.986473 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8gb2" event={"ID":"cb1dfcff-aa2f-477e-ab35-302441b374c3","Type":"ContainerStarted","Data":"a278c695b22e90f1a0fbdaae2154254927e3ce23bb2357e57ee2768aba91c0ee"} Jul 20 12:45:42.004296 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:42.004234 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m8gb2" podStartSLOduration=18.173496974 podStartE2EDuration="37.004213684s" totalImagesPullingTime="18.83071671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 12:45:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:42.003415808 +0000 UTC m=+83.975621642" watchObservedRunningTime="2026-07-20 12:45:42.004213684 +0000 UTC m=+83.976419516" Jul 20 12:45:42.055759 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:42.055717 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:42.808472 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:42.808437 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:42.906302 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:42.906263 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:43.616093 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:43.616044 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:43.713837 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:43.713800 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:43.813659 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:43.813623 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:44.163685 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:44.163602 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:44.265508 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:44.265468 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:44.835024 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:44.834981 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:44.912249 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:44.912189 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:44.990409 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:44.990350 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:45.050143 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:45.050110 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:48.454552 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:48.454517 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:48.510734 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:48.510699 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:48.595783 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:48.595754 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:48.976830 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:48.976795 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tlzx5" Jul 20 12:45:49.085670 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:49.085628 2570 patch_prober.go:28] interesting pod/image-registry-8f5ccc466-5ksdn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 12:45:49.085857 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:49.085689 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" podUID="b23c4f77-5615-4e72-ad77-2f228693db38" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 12:45:49.111992 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:49.111957 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:49.214060 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:49.214026 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:49.317316 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:49.317222 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:50.811330 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:50.811297 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:50.909835 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:50.909806 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8f5ccc466-5ksdn" Jul 20 12:45:51.029953 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:51.029878 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:51.077572 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:51.077489 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:51.156599 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:51.156562 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:51.262227 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:51.262196 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:52.031779 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:52.031725 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-hpsh8" event={"ID":"615ac5be-f68e-4381-a35a-5ce6ac9df15b","Type":"ContainerStarted","Data":"dcea6d17d1e40d6eb774f7e4a835ea2f9162189ddee140bb085e3dd93f29dc52"} Jul 20 12:45:52.032282 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:52.031982 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-hpsh8" Jul 20 12:45:52.046527 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:52.046486 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-hpsh8" Jul 20 12:45:52.049916 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:52.049848 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-hpsh8" podStartSLOduration=2.620649305 podStartE2EDuration="24.049826782s" totalImagesPullingTime="21.429177477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:52.049461853 +0000 UTC m=+94.021667688" watchObservedRunningTime="2026-07-20 12:45:52.049826782 +0000 UTC m=+94.022032617" Jul 20 12:45:52.307327 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:52.307236 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:52.544275 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:52.544221 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:52.970264 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:52.970218 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:53.193750 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.193712 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:53.237596 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.237514 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-spg2n"] Jul 20 12:45:53.286250 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.286215 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.286996 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.286887 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:53.288486 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.288461 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ft8lg\"" Jul 20 12:45:53.288705 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.288686 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 12:45:53.288939 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.288921 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 12:45:53.289118 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.289102 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 12:45:53.289479 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.289461 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 12:45:53.289663 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.289646 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 12:45:53.289901 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.289883 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 12:45:53.344246 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.344203 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-tls\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.344515 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.344498 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-wtmp\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.344607 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.344597 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-metrics-client-ca\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.344674 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.344663 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.344734 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.344724 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-accelerators-collector-config\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.344798 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.344790 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-sys\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.344869 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.344860 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-root\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.344936 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.344928 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-textfile\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.345057 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.345042 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm72f\" (UniqueName: \"kubernetes.io/projected/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-kube-api-access-vm72f\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.385821 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.385787 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:53.446576 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.446539 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-root\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.446772 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.446596 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-textfile\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.446772 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.446638 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm72f\" (UniqueName: \"kubernetes.io/projected/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-kube-api-access-vm72f\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.446772 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.446690 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-tls\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.446772 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.446715 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-wtmp\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.446772 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.446747 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-metrics-client-ca\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.447020 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.446791 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.447020 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.446822 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-accelerators-collector-config\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.447020 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.446853 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-sys\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.447020 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:53.446879 2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 20 12:45:53.447020 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.446926 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-sys\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.447020 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:53.446966 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-tls podName:a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d nodeName:}" failed. No retries permitted until 2026-07-20 12:45:53.946942911 +0000 UTC m=+95.919148747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-tls") pod "node-exporter-spg2n" (UID: "a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d") : secret "node-exporter-tls" not found Jul 20 12:45:53.447302 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.447050 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-textfile\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.447302 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.447063 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-wtmp\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.447611 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.447211 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-root\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.447704 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.447607 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-metrics-client-ca\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.447919 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.447873 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-accelerators-collector-config\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.450903 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.450870 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.488750 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.486189 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm72f\" (UniqueName: \"kubernetes.io/projected/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-kube-api-access-vm72f\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.490339 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.490279 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:53.587241 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.587203 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:53.616459 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.616224 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" podUID="24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" containerName="registry" containerID="cri-o://243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808" gracePeriod=30 Jul 20 12:45:53.661307 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.661268 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:53.725178 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.725101 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:53.929076 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.929039 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:53.953493 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.951607 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69lh6\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-kube-api-access-69lh6\") pod \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " Jul 20 12:45:53.953493 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.951682 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-certificates\") pod \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " Jul 20 12:45:53.953493 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.951779 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls\") pod \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " Jul 20 12:45:53.953493 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.951843 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-installation-pull-secrets\") pod \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " Jul 20 12:45:53.953493 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.951871 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-trusted-ca\") pod \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " Jul 20 12:45:53.953493 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.951916 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-ca-trust-extracted\") pod \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " Jul 20 12:45:53.953493 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.951951 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-image-registry-private-configuration\") pod \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " Jul 20 12:45:53.953493 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.952001 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-bound-sa-token\") pod \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\" (UID: \"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0\") " Jul 20 12:45:53.953493 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.952295 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-tls\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.953493 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.953105 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 12:45:53.958327 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.958152 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 12:45:53.958327 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.958211 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 12:45:53.966232 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.966194 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 12:45:53.967257 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.967200 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 12:45:53.968797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.968743 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d-node-exporter-tls\") pod \"node-exporter-spg2n\" (UID: \"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d\") " pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:53.969049 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.968959 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-kube-api-access-69lh6" (OuterVolumeSpecName: "kube-api-access-69lh6") pod "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0"). InnerVolumeSpecName "kube-api-access-69lh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 12:45:53.969451 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.969397 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 12:45:53.969616 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.969583 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" (UID: "24c7d17a-2831-40f6-a1cf-a07bb02fc3e0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 12:45:53.998384 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:53.998339 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:54.042190 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.042099 2570 generic.go:356] "Generic (PLEG): container finished" podID="24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" containerID="243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808" exitCode=0 Jul 20 12:45:54.042190 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.042155 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" event={"ID":"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0","Type":"ContainerDied","Data":"243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808"} Jul 20 12:45:54.042451 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.042249 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" event={"ID":"24c7d17a-2831-40f6-a1cf-a07bb02fc3e0","Type":"ContainerDied","Data":"fea8a7bfe203f1bf354c071d5ecb4c4384126b129a29ab3fa799206ae04d8113"} Jul 20 12:45:54.042451 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.042169 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fcfdc44d-flp46" Jul 20 12:45:54.042451 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.042279 2570 scope.go:117] "RemoveContainer" containerID="243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808" Jul 20 12:45:54.052443 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.052416 2570 scope.go:117] "RemoveContainer" containerID="243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808" Jul 20 12:45:54.052879 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:45:54.052796 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808\": container with ID starting with 243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808 not found: ID does not exist" containerID="243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808" Jul 20 12:45:54.052879 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.052832 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808"} err="failed to get container status \"243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808\": rpc error: code = NotFound desc = could not find container \"243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808\": container with ID starting with 243b1d44f52c86ee6e9eb2abef84b1509f303e440bb16ce95cc2bedda18a4808 not found: ID does not exist" Jul 20 12:45:54.053036 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.052899 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-bound-sa-token\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:45:54.053036 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.052914 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69lh6\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-kube-api-access-69lh6\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:45:54.053036 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.052931 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-certificates\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:45:54.053036 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.052945 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-registry-tls\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:45:54.053036 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.052958 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-installation-pull-secrets\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:45:54.053036 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.052968 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-trusted-ca\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:45:54.053036 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.052980 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-ca-trust-extracted\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:45:54.053036 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.052994 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0-image-registry-private-configuration\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:45:54.067991 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.067954 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fcfdc44d-flp46"] Jul 20 12:45:54.071377 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.071331 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6fcfdc44d-flp46"] Jul 20 12:45:54.203538 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.203508 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-spg2n" Jul 20 12:45:54.218291 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.218242 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:54.238835 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:45:54.238693 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48f2efa_fdc7_41b7_a4e3_1a7d5ddc8a5d.slice/crio-0ee3310bf906ee9cc9feb8d26a19f2201eb311dcf1dbeee9d5f420081f6a0a0d WatchSource:0}: Error finding container 0ee3310bf906ee9cc9feb8d26a19f2201eb311dcf1dbeee9d5f420081f6a0a0d: Status 404 returned error can't find the container with id 0ee3310bf906ee9cc9feb8d26a19f2201eb311dcf1dbeee9d5f420081f6a0a0d Jul 20 12:45:54.304197 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.304055 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:54.418177 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.418132 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:54.518337 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.518293 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:54.521081 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:54.521048 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" path="/var/lib/kubelet/pods/24c7d17a-2831-40f6-a1cf-a07bb02fc3e0/volumes" Jul 20 12:45:55.048654 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:55.048618 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spg2n" event={"ID":"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d","Type":"ContainerStarted","Data":"0ee3310bf906ee9cc9feb8d26a19f2201eb311dcf1dbeee9d5f420081f6a0a0d"} Jul 20 12:45:56.061200 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:56.061164 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spg2n" event={"ID":"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d","Type":"ContainerStarted","Data":"586a84cfd8dc08a583e5ce5831f3c744c6cf9be43a679a78562fd4a83baefaea"} Jul 20 12:45:56.183775 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:56.183731 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:56.244598 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:56.244544 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:56.622867 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:56.622831 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:56.883109 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:56.882974 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:57.067802 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:57.067759 2570 generic.go:356] "Generic (PLEG): container finished" podID="a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d" containerID="586a84cfd8dc08a583e5ce5831f3c744c6cf9be43a679a78562fd4a83baefaea" exitCode=0 Jul 20 12:45:57.068255 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:57.067851 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spg2n" event={"ID":"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d","Type":"ContainerDied","Data":"586a84cfd8dc08a583e5ce5831f3c744c6cf9be43a679a78562fd4a83baefaea"} Jul 20 12:45:58.075523 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:58.075481 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spg2n" event={"ID":"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d","Type":"ContainerStarted","Data":"35c917d2fd7de9d3e184f7a6002333e244cdf534513342818bce01bea6e68e55"} Jul 20 12:45:58.075992 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:58.075582 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spg2n" event={"ID":"a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d","Type":"ContainerStarted","Data":"a92f5ff5b0efcea5e7be2580f46cd67351287136052284cf0a41640ccc1d99a0"} Jul 20 12:45:58.096187 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:58.096123 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-spg2n" podStartSLOduration=3.437951279 podStartE2EDuration="5.096099599s" totalImagesPullingTime="1.557042279s" totalInitContainerRuntime="101.106041ms" isStatefulPod=true podCreationTimestamp="2026-07-20 12:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:45:58.095163903 +0000 UTC m=+100.067369758" watchObservedRunningTime="2026-07-20 12:45:58.096099599 +0000 UTC m=+100.068305432" Jul 20 12:45:58.180602 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:58.180558 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:58.281898 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:58.281861 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:58.579174 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:58.579135 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:58.850635 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:58.850592 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:58.901931 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:58.901887 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:59.454176 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.454139 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 12:45:59.454706 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.454538 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" containerName="registry" Jul 20 12:45:59.454706 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.454559 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" containerName="registry" Jul 20 12:45:59.454706 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.454643 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c7d17a-2831-40f6-a1cf-a07bb02fc3e0" containerName="registry" Jul 20 12:45:59.459877 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.459846 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:45:59.507200 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.507159 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 12:45:59.507396 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.507313 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.509728 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.509699 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 20 12:45:59.510001 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.509980 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-chcl5\"" Jul 20 12:45:59.510103 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.509983 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 20 12:45:59.510953 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.510627 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 20 12:45:59.510953 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.510655 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bbn22i2o7rfu9\"" Jul 20 12:45:59.510953 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.510668 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 20 12:45:59.510953 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.510634 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 20 12:45:59.510953 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.510773 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 20 12:45:59.511286 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.510955 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 20 12:45:59.511286 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.510957 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 20 12:45:59.511286 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.511063 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 20 12:45:59.511286 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.510963 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 20 12:45:59.511286 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.511098 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 20 12:45:59.511525 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.511346 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 20 12:45:59.513002 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.512062 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 20 12:45:59.513503 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.513449 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 20 12:45:59.516592 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.516567 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 20 12:45:59.603921 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.603880 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.603921 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.603925 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-web-config\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604148 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.603952 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-config-out\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604148 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.603977 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604148 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604054 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604148 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604105 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604148 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604141 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-config\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604358 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604197 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604358 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604261 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604358 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604295 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604358 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604324 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604358 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604340 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604358 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604380 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604686 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604417 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604686 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604463 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604686 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604518 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604686 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604552 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604686 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604593 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604686 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604618 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.604686 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.604674 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppk5h\" (UniqueName: \"kubernetes.io/projected/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-kube-api-access-ppk5h\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.705586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.705492 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.705586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.705550 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-config\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.705586 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.705582 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.705876 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.705611 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.705876 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.705640 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.705876 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.705668 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.705876 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.705691 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706264 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706228 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706360 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706276 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706360 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706313 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706360 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706348 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706548 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706409 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706548 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706439 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706548 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706446 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706548 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706468 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706548 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706523 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppk5h\" (UniqueName: \"kubernetes.io/projected/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-kube-api-access-ppk5h\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706807 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706573 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706807 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706602 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-web-config\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706807 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706653 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706807 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706661 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-config-out\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706807 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706707 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.706807 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.706756 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.707317 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.707290 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.709210 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.709178 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.709345 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.709187 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.709437 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.709420 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-config-out\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.709788 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.709755 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.709935 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.709910 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.710033 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.710005 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.711485 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.710649 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.711485 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.710649 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-config\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.712474 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.712421 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.712566 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.712498 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.713125 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.712956 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.713240 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.713215 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.714064 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.714021 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.714404 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.714353 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.714490 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.714390 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-web-config\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.715085 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.715057 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.717443 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.717423 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppk5h\" (UniqueName: \"kubernetes.io/projected/9fa8696b-a1e9-4ed5-bd40-02f7adcef946-kube-api-access-ppk5h\") pod \"prometheus-k8s-0\" (UID: \"9fa8696b-a1e9-4ed5-bd40-02f7adcef946\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:45:59.820651 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:45:59.820605 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:46:00.017601 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:00.017408 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 12:46:00.023881 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:46:00.023833 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa8696b_a1e9_4ed5_bd40_02f7adcef946.slice/crio-691f1e30baec6b9cf145ed409f1c5c51f83bb06aed63b6c1786d16c882b53dea WatchSource:0}: Error finding container 691f1e30baec6b9cf145ed409f1c5c51f83bb06aed63b6c1786d16c882b53dea: Status 404 returned error can't find the container with id 691f1e30baec6b9cf145ed409f1c5c51f83bb06aed63b6c1786d16c882b53dea Jul 20 12:46:00.086293 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:00.086254 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fa8696b-a1e9-4ed5-bd40-02f7adcef946","Type":"ContainerStarted","Data":"691f1e30baec6b9cf145ed409f1c5c51f83bb06aed63b6c1786d16c882b53dea"} Jul 20 12:46:00.908178 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:00.908146 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-67vbc" Jul 20 12:46:01.430602 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:01.430569 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:01.532322 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:01.532287 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:01.631826 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:01.631796 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:02.002463 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:02.002363 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:02.095598 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:02.095562 2570 generic.go:356] "Generic (PLEG): container finished" podID="9fa8696b-a1e9-4ed5-bd40-02f7adcef946" containerID="df8b7775a62f49fe3b688ff066e398c217f48206f99134b997ebb8dbc7b5acf0" exitCode=0 Jul 20 12:46:02.095760 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:02.095622 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fa8696b-a1e9-4ed5-bd40-02f7adcef946","Type":"ContainerDied","Data":"df8b7775a62f49fe3b688ff066e398c217f48206f99134b997ebb8dbc7b5acf0"} Jul 20 12:46:02.103548 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:02.103522 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:06.322692 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:06.322605 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:06.422804 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:06.422762 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:06.523932 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:06.523899 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:07.119124 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:07.119085 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fa8696b-a1e9-4ed5-bd40-02f7adcef946","Type":"ContainerStarted","Data":"02c32359fc50793b35671d44ae9fa7fb9a2809248e854d59bf2d991767aaf2a4"} Jul 20 12:46:07.119439 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:07.119163 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fa8696b-a1e9-4ed5-bd40-02f7adcef946","Type":"ContainerStarted","Data":"917af12c5aaf1795d60eb496a1a834b84bacbff768b383c3f4966d5acfbf8a80"} Jul 20 12:46:07.140558 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:07.140518 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:07.241322 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:07.241290 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:07.908579 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:07.908539 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:08.008161 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:08.008120 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:09.130358 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:09.130319 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fa8696b-a1e9-4ed5-bd40-02f7adcef946","Type":"ContainerStarted","Data":"441508c5278efde682b95a33348b11610bc089e306fac53a7718332c64aa6def"} Jul 20 12:46:09.130879 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:09.130406 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fa8696b-a1e9-4ed5-bd40-02f7adcef946","Type":"ContainerStarted","Data":"ed5890ee9da3963d235f07c1e010dd634770640d3458926a8a320f4bcbdbd791"} Jul 20 12:46:09.130879 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:09.130441 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fa8696b-a1e9-4ed5-bd40-02f7adcef946","Type":"ContainerStarted","Data":"2e22f05948b47f448aa4844fc9abc4d0b5a5fc1a1a410a0b276cacbeaafd564e"} Jul 20 12:46:09.130879 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:09.130466 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fa8696b-a1e9-4ed5-bd40-02f7adcef946","Type":"ContainerStarted","Data":"8ba0f5e6aac4cfd8cff930617ff4664c85b67f838965ee180540acab051a0f3e"} Jul 20 12:46:09.158441 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:09.158346 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.21088034 podStartE2EDuration="10.158328778s" totalImagesPullingTime="7.924814968s" totalInitContainerRuntime="22.63347ms" isStatefulPod=false podCreationTimestamp="2026-07-20 12:45:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:46:09.15673896 +0000 UTC m=+111.128944791" watchObservedRunningTime="2026-07-20 12:46:09.158328778 +0000 UTC m=+111.130534610" Jul 20 12:46:09.821574 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:09.821533 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:46:11.063604 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:11.063556 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:11.162619 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:11.162576 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:13.685532 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:13.685498 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:13.735048 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:13.735015 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:16.551464 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:16.551431 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:16.656110 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:16.656076 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:16.717510 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:16.717475 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:22.182054 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:22.182018 2570 generic.go:356] "Generic (PLEG): container finished" podID="ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca" containerID="1b3460ac134ea08d277ea96b4dea9e2bd8d168c434f3d8f12418e0088300c112" exitCode=0 Jul 20 12:46:22.182505 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:22.182092 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" event={"ID":"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca","Type":"ContainerDied","Data":"1b3460ac134ea08d277ea96b4dea9e2bd8d168c434f3d8f12418e0088300c112"} Jul 20 12:46:22.182505 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:22.182426 2570 scope.go:117] "RemoveContainer" containerID="1b3460ac134ea08d277ea96b4dea9e2bd8d168c434f3d8f12418e0088300c112" Jul 20 12:46:23.188208 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:23.188175 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zwnsq" event={"ID":"ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca","Type":"ContainerStarted","Data":"4f5654bbb876c96a8e89c4ac4839390be491f19669401c596bf3ad557a4057b7"} Jul 20 12:46:26.797437 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:26.797399 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:28.677701 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:28.677666 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:28.740607 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:28.740576 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:37.353302 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:37.353268 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:46:59.821155 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:59.821110 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:46:59.837987 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:46:59.837945 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:47:00.347558 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:00.347529 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 12:47:09.340698 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:09.340662 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:47:09.403673 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:09.403635 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:47:19.491543 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:19.491460 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:47:19.549068 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:19.549031 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:47:21.245877 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:21.245839 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:47:21.304532 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:21.304497 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:47:28.686316 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:28.686280 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:47:28.788572 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:28.788533 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:47:30.950958 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:30.950924 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:47:31.051026 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:31.050991 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:47:56.991157 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:56.991121 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:47:57.041790 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:47:57.041761 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:18.187267 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:18.187213 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:18.707422 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:18.707386 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:18.766610 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:18.766574 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:26.727180 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:26.727122 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:26.827743 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:26.827702 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:40.015149 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:40.015078 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:41.413358 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:41.413298 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:43.651552 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:43.651517 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:43.751655 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:43.751624 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:45.418728 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:45.418697 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:45.542480 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:45.542448 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:45.681204 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:45.681085 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:47.965053 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:47.965026 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:48.180725 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:48.180652 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:50.618835 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:50.618792 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:50.677741 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:50.677685 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:55.178409 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:55.178347 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:57.424155 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:57.424098 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:48:57.484971 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:48:57.484936 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:03.190751 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:03.190722 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:03.292441 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:03.292360 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:05.381154 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:05.381082 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:09.175860 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:09.175804 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:09.275495 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:09.275443 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:18.450959 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:18.450926 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 12:49:18.457626 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:18.457589 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 12:49:18.461169 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:18.461135 2570 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 12:49:21.695089 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.695036 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:21.820281 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.820242 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n"] Jul 20 12:49:21.822321 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.822301 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.824989 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.824960 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 20 12:49:21.825150 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.825126 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 20 12:49:21.825330 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.825310 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 20 12:49:21.825641 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.825624 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 20 12:49:21.825840 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.825824 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 20 12:49:21.826043 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.826028 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k7fgz\"" Jul 20 12:49:21.841445 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.841404 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n"] Jul 20 12:49:21.853555 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.853521 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:21.876560 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.876525 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7055925-99db-41ec-a730-5e633ce939b2-metrics-cert\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.876560 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.876567 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7055925-99db-41ec-a730-5e633ce939b2-cert\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.876777 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.876595 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn2xx\" (UniqueName: \"kubernetes.io/projected/a7055925-99db-41ec-a730-5e633ce939b2-kube-api-access-sn2xx\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.876777 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.876723 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a7055925-99db-41ec-a730-5e633ce939b2-manager-config\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.977992 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.977906 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7055925-99db-41ec-a730-5e633ce939b2-metrics-cert\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.977992 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.977942 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7055925-99db-41ec-a730-5e633ce939b2-cert\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.977992 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.977969 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn2xx\" (UniqueName: \"kubernetes.io/projected/a7055925-99db-41ec-a730-5e633ce939b2-kube-api-access-sn2xx\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.978268 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.978017 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a7055925-99db-41ec-a730-5e633ce939b2-manager-config\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.978778 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.978750 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a7055925-99db-41ec-a730-5e633ce939b2-manager-config\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.980527 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.980508 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7055925-99db-41ec-a730-5e633ce939b2-metrics-cert\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.980682 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.980662 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7055925-99db-41ec-a730-5e633ce939b2-cert\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:21.988125 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:21.988097 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn2xx\" (UniqueName: \"kubernetes.io/projected/a7055925-99db-41ec-a730-5e633ce939b2-kube-api-access-sn2xx\") pod \"lws-controller-manager-586f8df4c4-jz65n\" (UID: \"a7055925-99db-41ec-a730-5e633ce939b2\") " pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:22.132092 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:22.132054 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:22.277882 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:22.277804 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n"] Jul 20 12:49:22.280775 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:49:22.280746 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7055925_99db_41ec_a730_5e633ce939b2.slice/crio-5833b04903190fe78160b1caec94b4b35fa8e6d7a259999bbe14be6afadc05ca WatchSource:0}: Error finding container 5833b04903190fe78160b1caec94b4b35fa8e6d7a259999bbe14be6afadc05ca: Status 404 returned error can't find the container with id 5833b04903190fe78160b1caec94b4b35fa8e6d7a259999bbe14be6afadc05ca Jul 20 12:49:22.282841 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:22.282823 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 12:49:22.865396 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:22.865334 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" event={"ID":"a7055925-99db-41ec-a730-5e633ce939b2","Type":"ContainerStarted","Data":"5833b04903190fe78160b1caec94b4b35fa8e6d7a259999bbe14be6afadc05ca"} Jul 20 12:49:23.029751 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:23.029697 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:23.139233 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:23.139099 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:23.615508 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:23.615473 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:23.986344 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:23.986232 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:24.086525 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.086492 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:24.124485 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.124283 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7"] Jul 20 12:49:24.131928 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.131893 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.134430 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.134398 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 12:49:24.134613 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.134540 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 12:49:24.134702 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.134664 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 12:49:24.134758 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.134722 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5cnmf\"" Jul 20 12:49:24.136086 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.136062 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 12:49:24.144875 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.144847 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7"] Jul 20 12:49:24.197835 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.197792 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7bfa49f3-e344-4930-8b09-0edb79e8c08b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.198029 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.197846 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7bfa49f3-e344-4930-8b09-0edb79e8c08b-manifests\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.198029 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.197916 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vgvd\" (UniqueName: \"kubernetes.io/projected/7bfa49f3-e344-4930-8b09-0edb79e8c08b-kube-api-access-4vgvd\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.198029 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.197976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bfa49f3-e344-4930-8b09-0edb79e8c08b-webhook-cert\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.198188 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.198058 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bfa49f3-e344-4930-8b09-0edb79e8c08b-tmp\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.249415 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.249312 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:24.299135 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.299091 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bfa49f3-e344-4930-8b09-0edb79e8c08b-tmp\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.299319 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.299195 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7bfa49f3-e344-4930-8b09-0edb79e8c08b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.299319 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.299242 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7bfa49f3-e344-4930-8b09-0edb79e8c08b-manifests\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.299319 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.299267 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vgvd\" (UniqueName: \"kubernetes.io/projected/7bfa49f3-e344-4930-8b09-0edb79e8c08b-kube-api-access-4vgvd\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.299319 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.299299 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bfa49f3-e344-4930-8b09-0edb79e8c08b-webhook-cert\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.299685 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.299659 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7bfa49f3-e344-4930-8b09-0edb79e8c08b-manifests\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.302051 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.302022 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bfa49f3-e344-4930-8b09-0edb79e8c08b-tmp\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.302263 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.302247 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7bfa49f3-e344-4930-8b09-0edb79e8c08b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.302400 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.302349 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bfa49f3-e344-4930-8b09-0edb79e8c08b-webhook-cert\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.311711 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.311676 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vgvd\" (UniqueName: \"kubernetes.io/projected/7bfa49f3-e344-4930-8b09-0edb79e8c08b-kube-api-access-4vgvd\") pod \"opendatahub-operator-controller-manager-749978ccd7-kpss7\" (UID: \"7bfa49f3-e344-4930-8b09-0edb79e8c08b\") " pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.351286 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.351243 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:24.445565 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.445531 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:24.456465 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.456436 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:24.577517 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.577480 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:24.678327 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.678297 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:24.699345 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.699311 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7"] Jul 20 12:49:24.702579 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:49:24.702533 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bfa49f3_e344_4930_8b09_0edb79e8c08b.slice/crio-a1bde42975008b414f0179611197fe25a729e0d1f5bd9054649b489de4db2502 WatchSource:0}: Error finding container a1bde42975008b414f0179611197fe25a729e0d1f5bd9054649b489de4db2502: Status 404 returned error can't find the container with id a1bde42975008b414f0179611197fe25a729e0d1f5bd9054649b489de4db2502 Jul 20 12:49:24.783546 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.783507 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:24.876545 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.876508 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" event={"ID":"7bfa49f3-e344-4930-8b09-0edb79e8c08b","Type":"ContainerStarted","Data":"a1bde42975008b414f0179611197fe25a729e0d1f5bd9054649b489de4db2502"} Jul 20 12:49:24.877856 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.877832 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" event={"ID":"a7055925-99db-41ec-a730-5e633ce939b2","Type":"ContainerStarted","Data":"dc358a49b290ada64e5e6a1a0510641f5499cadf8933859ac199e674737d0978"} Jul 20 12:49:24.878074 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.878058 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:24.902661 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:24.902611 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" podStartSLOduration=1.588072304 podStartE2EDuration="3.902593303s" totalImagesPullingTime="2.314520999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:49:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:49:24.901739004 +0000 UTC m=+306.873944836" watchObservedRunningTime="2026-07-20 12:49:24.902593303 +0000 UTC m=+306.874799134" Jul 20 12:49:27.894487 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:27.894444 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" event={"ID":"7bfa49f3-e344-4930-8b09-0edb79e8c08b","Type":"ContainerStarted","Data":"6c77587bec6a11232e296a132a1f16099b0788a8d17785e491bd2a80caf6d0ba"} Jul 20 12:49:28.900130 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:28.900094 2570 generic.go:356] "Generic (PLEG): container finished" podID="7bfa49f3-e344-4930-8b09-0edb79e8c08b" containerID="6c77587bec6a11232e296a132a1f16099b0788a8d17785e491bd2a80caf6d0ba" exitCode=0 Jul 20 12:49:28.900527 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:28.900183 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" event={"ID":"7bfa49f3-e344-4930-8b09-0edb79e8c08b","Type":"ContainerDied","Data":"6c77587bec6a11232e296a132a1f16099b0788a8d17785e491bd2a80caf6d0ba"} Jul 20 12:49:29.907453 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:29.907417 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" event={"ID":"7bfa49f3-e344-4930-8b09-0edb79e8c08b","Type":"ContainerStarted","Data":"541c326b2f5ab339d4dc845e2ef5fe9b8bf84865f4819be56184e94cf97b9bcf"} Jul 20 12:49:29.907834 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:29.907555 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:29.930099 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:29.930046 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" podStartSLOduration=1.569914321 podStartE2EDuration="5.930026847s" totalImagesPullingTime="3.30757096s" totalInitContainerRuntime="1.052541566s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:49:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:49:29.929278926 +0000 UTC m=+311.901484758" watchObservedRunningTime="2026-07-20 12:49:29.930026847 +0000 UTC m=+311.902232743" Jul 20 12:49:35.886117 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:35.886086 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-586f8df4c4-jz65n" Jul 20 12:49:40.914745 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:40.914712 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-749978ccd7-kpss7" Jul 20 12:49:42.846243 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:42.846210 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:44.260108 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:44.260070 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:44.360640 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:44.360606 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:53.898312 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:53.898242 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:54.027441 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:54.027402 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:54.141999 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:54.141966 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:54.433425 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:54.433355 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:54.521304 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:54.521273 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:55.373756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:55.373725 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:55.490598 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:55.490561 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:55.644149 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:55.644045 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:55.782648 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:55.782620 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:49:56.946212 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:49:56.946177 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:00.780078 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:00.780036 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:00.880471 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:00.880437 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:01.053282 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.053196 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5"] Jul 20 12:50:01.062545 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.062514 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.064540 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.064515 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 12:50:01.064540 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.064529 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-mkvgz\"" Jul 20 12:50:01.070199 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.070171 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5"] Jul 20 12:50:01.084653 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.084622 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:01.106927 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.106876 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82j4q\" (UniqueName: \"kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-kube-api-access-82j4q\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.107091 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.106931 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.107091 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.106988 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.107091 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.107033 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.107199 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.107092 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.107199 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.107111 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.107199 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.107164 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.107291 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.107199 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.107291 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.107230 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.156185 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.156092 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw"] Jul 20 12:50:01.159276 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.159245 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.178350 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.178315 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw"] Jul 20 12:50:01.207811 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.207769 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.207811 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.207812 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82j4q\" (UniqueName: \"kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-kube-api-access-82j4q\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.208077 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.207831 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.208077 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208011 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.208077 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208065 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.208225 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208107 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.208225 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208134 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.208225 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208161 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.208225 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208190 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.208225 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208198 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.208562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208259 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.208562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208290 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.208562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.208562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208384 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.208562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208416 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.208562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208459 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.208562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208456 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.208562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208489 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.208562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208555 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.208988 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208591 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cng8z\" (UniqueName: \"kubernetes.io/projected/01a555a6-3312-4a51-a13c-2fad0b03fe1a-kube-api-access-cng8z\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.208988 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208650 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.208988 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.208745 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.209149 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.209073 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.210642 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.210620 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.210785 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.210762 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.216822 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.216793 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.217067 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.217043 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82j4q\" (UniqueName: \"kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-kube-api-access-82j4q\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-lflk5\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.309313 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.309207 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.309313 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.309273 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.309313 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.309306 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.309573 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.309332 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.309573 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.309493 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.309573 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.309542 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.309715 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.309590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.309715 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.309636 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.309715 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.309683 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cng8z\" (UniqueName: \"kubernetes.io/projected/01a555a6-3312-4a51-a13c-2fad0b03fe1a-kube-api-access-cng8z\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.309863 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.309832 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.310069 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.310044 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.310181 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.310097 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.310247 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.310229 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.310333 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.310314 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.312070 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.312046 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.312190 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.312122 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.324918 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.324880 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cng8z\" (UniqueName: \"kubernetes.io/projected/01a555a6-3312-4a51-a13c-2fad0b03fe1a-kube-api-access-cng8z\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.325198 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.325175 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a555a6-3312-4a51-a13c-2fad0b03fe1a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf482zmw\" (UID: \"01a555a6-3312-4a51-a13c-2fad0b03fe1a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.344077 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.344043 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:01.376093 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.376059 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:01.471227 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.471194 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:01.536532 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.536505 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5"] Jul 20 12:50:01.538854 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:50:01.538823 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ab39eb_d512_4ebe_b726_9adddb7b5d64.slice/crio-db3c6d457b7fa09109563898a40fefb74cc6fb4aa558888a0cee6a0493b13c2a WatchSource:0}: Error finding container db3c6d457b7fa09109563898a40fefb74cc6fb4aa558888a0cee6a0493b13c2a: Status 404 returned error can't find the container with id db3c6d457b7fa09109563898a40fefb74cc6fb4aa558888a0cee6a0493b13c2a Jul 20 12:50:01.628433 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.628397 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw"] Jul 20 12:50:01.631232 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:50:01.631205 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a555a6_3312_4a51_a13c_2fad0b03fe1a.slice/crio-6b8c1f2aa2da1e07c6c95ba865fa0396aeab0cb9a988ae0f94270b8603f3d339 WatchSource:0}: Error finding container 6b8c1f2aa2da1e07c6c95ba865fa0396aeab0cb9a988ae0f94270b8603f3d339: Status 404 returned error can't find the container with id 6b8c1f2aa2da1e07c6c95ba865fa0396aeab0cb9a988ae0f94270b8603f3d339 Jul 20 12:50:01.665330 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:01.665295 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:02.040832 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:02.040797 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" event={"ID":"01a555a6-3312-4a51-a13c-2fad0b03fe1a","Type":"ContainerStarted","Data":"6b8c1f2aa2da1e07c6c95ba865fa0396aeab0cb9a988ae0f94270b8603f3d339"} Jul 20 12:50:02.042051 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:02.042028 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" event={"ID":"37ab39eb-d512-4ebe-b726-9adddb7b5d64","Type":"ContainerStarted","Data":"db3c6d457b7fa09109563898a40fefb74cc6fb4aa558888a0cee6a0493b13c2a"} Jul 20 12:50:04.886788 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:04.886744 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 12:50:04.887086 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:04.886826 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 12:50:04.887086 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:04.886858 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 12:50:05.059838 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:05.059788 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" event={"ID":"01a555a6-3312-4a51-a13c-2fad0b03fe1a","Type":"ContainerStarted","Data":"ac45dc1c24d1d93f10bb80e3f61120332b290927692fbc267d48d1c36ed6b691"} Jul 20 12:50:05.079568 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:05.079495 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" podStartSLOduration=0.826278685 podStartE2EDuration="4.079469072s" totalImagesPullingTime="3.253190387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:50:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:50:05.078348015 +0000 UTC m=+347.050553848" watchObservedRunningTime="2026-07-20 12:50:05.079469072 +0000 UTC m=+347.051674906" Jul 20 12:50:05.472074 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:05.472038 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:05.544885 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:05.544855 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:06.063973 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:06.063944 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:06.065090 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:06.065069 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf482zmw" Jul 20 12:50:06.116592 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:06.116556 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5"] Jul 20 12:50:06.602145 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:06.602109 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:06.664543 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:06.664511 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:08.550268 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:08.550234 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:15.524182 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:15.524145 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 12:50:15.524474 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:15.524225 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 12:50:15.524474 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:15.524255 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 12:50:16.106095 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:16.106058 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" event={"ID":"37ab39eb-d512-4ebe-b726-9adddb7b5d64","Type":"ContainerStarted","Data":"64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f"} Jul 20 12:50:16.106288 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:16.106145 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" podUID="37ab39eb-d512-4ebe-b726-9adddb7b5d64" containerName="istio-proxy" containerID="cri-o://64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f" gracePeriod=30 Jul 20 12:50:16.129804 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:16.129747 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" podStartSLOduration=1.14658559 podStartE2EDuration="15.129725557s" totalImagesPullingTime="13.983139967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:50:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:50:16.127986461 +0000 UTC m=+358.100192293" watchObservedRunningTime="2026-07-20 12:50:16.129725557 +0000 UTC m=+358.101931451" Jul 20 12:50:16.376822 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:16.376717 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:21.348499 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.348471 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:21.492149 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492057 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-token\") pod \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " Jul 20 12:50:21.492149 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492109 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-envoy\") pod \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " Jul 20 12:50:21.492149 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492143 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-certs\") pod \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " Jul 20 12:50:21.492401 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492176 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82j4q\" (UniqueName: \"kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-kube-api-access-82j4q\") pod \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " Jul 20 12:50:21.492401 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492258 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-credential-socket\") pod \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " Jul 20 12:50:21.492401 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492296 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-data\") pod \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " Jul 20 12:50:21.492401 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492318 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istiod-ca-cert\") pod \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " Jul 20 12:50:21.492401 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492385 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-podinfo\") pod \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " Jul 20 12:50:21.492655 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492425 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-socket\") pod \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\" (UID: \"37ab39eb-d512-4ebe-b726-9adddb7b5d64\") " Jul 20 12:50:21.492655 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492494 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37ab39eb-d512-4ebe-b726-9adddb7b5d64" (UID: "37ab39eb-d512-4ebe-b726-9adddb7b5d64"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 12:50:21.492757 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492653 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37ab39eb-d512-4ebe-b726-9adddb7b5d64" (UID: "37ab39eb-d512-4ebe-b726-9adddb7b5d64"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 12:50:21.492757 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492742 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37ab39eb-d512-4ebe-b726-9adddb7b5d64" (UID: "37ab39eb-d512-4ebe-b726-9adddb7b5d64"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 12:50:21.492858 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492761 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-data" (OuterVolumeSpecName: "istio-data") pod "37ab39eb-d512-4ebe-b726-9adddb7b5d64" (UID: "37ab39eb-d512-4ebe-b726-9adddb7b5d64"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 12:50:21.492858 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492819 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-socket\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:50:21.492858 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492836 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-workload-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:50:21.492858 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.492849 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-credential-socket\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:50:21.493051 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.493017 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37ab39eb-d512-4ebe-b726-9adddb7b5d64" (UID: "37ab39eb-d512-4ebe-b726-9adddb7b5d64"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 12:50:21.495052 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.495014 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37ab39eb-d512-4ebe-b726-9adddb7b5d64" (UID: "37ab39eb-d512-4ebe-b726-9adddb7b5d64"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 12:50:21.495052 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.495027 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-token" (OuterVolumeSpecName: "istio-token") pod "37ab39eb-d512-4ebe-b726-9adddb7b5d64" (UID: "37ab39eb-d512-4ebe-b726-9adddb7b5d64"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 12:50:21.495236 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.495102 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37ab39eb-d512-4ebe-b726-9adddb7b5d64" (UID: "37ab39eb-d512-4ebe-b726-9adddb7b5d64"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 12:50:21.495236 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.495137 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-kube-api-access-82j4q" (OuterVolumeSpecName: "kube-api-access-82j4q") pod "37ab39eb-d512-4ebe-b726-9adddb7b5d64" (UID: "37ab39eb-d512-4ebe-b726-9adddb7b5d64"). InnerVolumeSpecName "kube-api-access-82j4q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 12:50:21.593848 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.593816 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-data\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:50:21.593848 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.593844 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istiod-ca-cert\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:50:21.593848 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.593853 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-podinfo\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:50:21.594076 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.593861 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-token\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:50:21.594076 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.593870 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37ab39eb-d512-4ebe-b726-9adddb7b5d64-istio-envoy\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:50:21.594076 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:21.593878 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82j4q\" (UniqueName: \"kubernetes.io/projected/37ab39eb-d512-4ebe-b726-9adddb7b5d64-kube-api-access-82j4q\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:50:22.132045 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:22.132005 2570 generic.go:356] "Generic (PLEG): container finished" podID="37ab39eb-d512-4ebe-b726-9adddb7b5d64" containerID="64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f" exitCode=0 Jul 20 12:50:22.132257 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:22.132054 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" event={"ID":"37ab39eb-d512-4ebe-b726-9adddb7b5d64","Type":"ContainerDied","Data":"64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f"} Jul 20 12:50:22.132257 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:22.132112 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" event={"ID":"37ab39eb-d512-4ebe-b726-9adddb7b5d64","Type":"ContainerDied","Data":"db3c6d457b7fa09109563898a40fefb74cc6fb4aa558888a0cee6a0493b13c2a"} Jul 20 12:50:22.132257 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:22.132135 2570 scope.go:117] "RemoveContainer" containerID="64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f" Jul 20 12:50:22.132257 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:22.132068 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5" Jul 20 12:50:22.141261 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:22.141232 2570 scope.go:117] "RemoveContainer" containerID="64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f" Jul 20 12:50:22.141541 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:50:22.141521 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f\": container with ID starting with 64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f not found: ID does not exist" containerID="64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f" Jul 20 12:50:22.141595 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:22.141551 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f"} err="failed to get container status \"64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f\": rpc error: code = NotFound desc = could not find container \"64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f\": container with ID starting with 64569f1c9dcbd791f0463a8e35c097a7dc9ad0be54a6e3a57fa53c3d6bf7788f not found: ID does not exist" Jul 20 12:50:22.153223 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:22.153192 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5"] Jul 20 12:50:22.156971 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:22.156943 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-lflk5"] Jul 20 12:50:22.521234 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:22.521137 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37ab39eb-d512-4ebe-b726-9adddb7b5d64" path="/var/lib/kubelet/pods/37ab39eb-d512-4ebe-b726-9adddb7b5d64/volumes" Jul 20 12:50:44.369943 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.369907 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:44.520899 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.520865 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:44.560403 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.560330 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-77zq7"] Jul 20 12:50:44.561164 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.561140 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37ab39eb-d512-4ebe-b726-9adddb7b5d64" containerName="istio-proxy" Jul 20 12:50:44.561327 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.561315 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ab39eb-d512-4ebe-b726-9adddb7b5d64" containerName="istio-proxy" Jul 20 12:50:44.561537 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.561523 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="37ab39eb-d512-4ebe-b726-9adddb7b5d64" containerName="istio-proxy" Jul 20 12:50:44.571869 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.571841 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-77zq7" Jul 20 12:50:44.575042 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.574861 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 12:50:44.575485 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.575303 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-mf6x4\"" Jul 20 12:50:44.576849 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.576822 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-77zq7"] Jul 20 12:50:44.578324 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.577837 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 12:50:44.616227 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.616184 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2zjf\" (UniqueName: \"kubernetes.io/projected/4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa-kube-api-access-q2zjf\") pod \"kuadrant-operator-catalog-77zq7\" (UID: \"4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa\") " pod="kuadrant-system/kuadrant-operator-catalog-77zq7" Jul 20 12:50:44.683489 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.683408 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:50:44.716787 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.716746 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2zjf\" (UniqueName: \"kubernetes.io/projected/4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa-kube-api-access-q2zjf\") pod \"kuadrant-operator-catalog-77zq7\" (UID: \"4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa\") " pod="kuadrant-system/kuadrant-operator-catalog-77zq7" Jul 20 12:50:44.729604 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.729572 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2zjf\" (UniqueName: \"kubernetes.io/projected/4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa-kube-api-access-q2zjf\") pod \"kuadrant-operator-catalog-77zq7\" (UID: \"4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa\") " pod="kuadrant-system/kuadrant-operator-catalog-77zq7" Jul 20 12:50:44.844494 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.844460 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-77zq7"] Jul 20 12:50:44.844757 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.844737 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-77zq7" Jul 20 12:50:44.993730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:44.993645 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-77zq7"] Jul 20 12:50:44.997169 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:50:44.997140 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea30ee3_42e6_4d7b_9300_3cdbc89b7aaa.slice/crio-c6ece3f3b48389422dd56c6209b86a746ce81bfe570d850631d0b24e5abd6757 WatchSource:0}: Error finding container c6ece3f3b48389422dd56c6209b86a746ce81bfe570d850631d0b24e5abd6757: Status 404 returned error can't find the container with id c6ece3f3b48389422dd56c6209b86a746ce81bfe570d850631d0b24e5abd6757 Jul 20 12:50:45.071582 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:45.071539 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t86cr"] Jul 20 12:50:45.076169 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:45.076139 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-t86cr" Jul 20 12:50:45.120103 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:45.120065 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grkgm\" (UniqueName: \"kubernetes.io/projected/62d8932f-d2d1-4fcd-9f2b-8308ab8174e7-kube-api-access-grkgm\") pod \"kuadrant-operator-catalog-t86cr\" (UID: \"62d8932f-d2d1-4fcd-9f2b-8308ab8174e7\") " pod="kuadrant-system/kuadrant-operator-catalog-t86cr" Jul 20 12:50:45.120602 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:45.120574 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t86cr"] Jul 20 12:50:45.221508 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:45.221462 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grkgm\" (UniqueName: \"kubernetes.io/projected/62d8932f-d2d1-4fcd-9f2b-8308ab8174e7-kube-api-access-grkgm\") pod \"kuadrant-operator-catalog-t86cr\" (UID: \"62d8932f-d2d1-4fcd-9f2b-8308ab8174e7\") " pod="kuadrant-system/kuadrant-operator-catalog-t86cr" Jul 20 12:50:45.233184 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:45.233130 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-77zq7" event={"ID":"4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa","Type":"ContainerStarted","Data":"c6ece3f3b48389422dd56c6209b86a746ce81bfe570d850631d0b24e5abd6757"} Jul 20 12:50:45.263957 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:45.263881 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grkgm\" (UniqueName: \"kubernetes.io/projected/62d8932f-d2d1-4fcd-9f2b-8308ab8174e7-kube-api-access-grkgm\") pod \"kuadrant-operator-catalog-t86cr\" (UID: \"62d8932f-d2d1-4fcd-9f2b-8308ab8174e7\") " pod="kuadrant-system/kuadrant-operator-catalog-t86cr" Jul 20 12:50:45.385314 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:45.385268 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-t86cr" Jul 20 12:50:45.543927 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:45.543902 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t86cr"] Jul 20 12:50:45.546472 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:50:45.546441 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d8932f_d2d1_4fcd_9f2b_8308ab8174e7.slice/crio-dfbb427cbbedf3329141b3d41007637e7568d33bb093b78be2be300ebb89d055 WatchSource:0}: Error finding container dfbb427cbbedf3329141b3d41007637e7568d33bb093b78be2be300ebb89d055: Status 404 returned error can't find the container with id dfbb427cbbedf3329141b3d41007637e7568d33bb093b78be2be300ebb89d055 Jul 20 12:50:46.245616 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:46.245561 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-t86cr" event={"ID":"62d8932f-d2d1-4fcd-9f2b-8308ab8174e7","Type":"ContainerStarted","Data":"dfbb427cbbedf3329141b3d41007637e7568d33bb093b78be2be300ebb89d055"} Jul 20 12:50:48.255997 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:48.255962 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-t86cr" event={"ID":"62d8932f-d2d1-4fcd-9f2b-8308ab8174e7","Type":"ContainerStarted","Data":"57f4c90d2f407df45d77177a4c8f5d1ace74ac46502b4d8d9265db77937d0bdd"} Jul 20 12:50:48.257439 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:48.257417 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-77zq7" event={"ID":"4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa","Type":"ContainerStarted","Data":"31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b"} Jul 20 12:50:48.257516 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:48.257494 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-77zq7" podUID="4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa" containerName="registry-server" containerID="cri-o://31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b" gracePeriod=2 Jul 20 12:50:48.272727 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:48.272671 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-t86cr" podStartSLOduration=1.543356539 podStartE2EDuration="3.272650238s" totalImagesPullingTime="1.729293699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:50:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:50:48.272356477 +0000 UTC m=+390.244562310" watchObservedRunningTime="2026-07-20 12:50:48.272650238 +0000 UTC m=+390.244856070" Jul 20 12:50:48.292021 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:48.291961 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-77zq7" podStartSLOduration=2.016922423 podStartE2EDuration="4.291942653s" totalImagesPullingTime="2.27502023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:50:48.291640885 +0000 UTC m=+390.263846717" watchObservedRunningTime="2026-07-20 12:50:48.291942653 +0000 UTC m=+390.264148484" Jul 20 12:50:48.509009 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:48.508938 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-77zq7" Jul 20 12:50:48.654065 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:48.654026 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2zjf\" (UniqueName: \"kubernetes.io/projected/4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa-kube-api-access-q2zjf\") pod \"4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa\" (UID: \"4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa\") " Jul 20 12:50:48.656448 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:48.656421 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa-kube-api-access-q2zjf" (OuterVolumeSpecName: "kube-api-access-q2zjf") pod "4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa" (UID: "4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa"). InnerVolumeSpecName "kube-api-access-q2zjf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 12:50:48.755572 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:48.755524 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2zjf\" (UniqueName: \"kubernetes.io/projected/4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa-kube-api-access-q2zjf\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 12:50:49.264043 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:49.264010 2570 generic.go:356] "Generic (PLEG): container finished" podID="4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa" containerID="31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b" exitCode=0 Jul 20 12:50:49.264486 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:49.264053 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-77zq7" Jul 20 12:50:49.264486 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:49.264095 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-77zq7" event={"ID":"4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa","Type":"ContainerDied","Data":"31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b"} Jul 20 12:50:49.264486 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:49.264136 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-77zq7" event={"ID":"4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa","Type":"ContainerDied","Data":"c6ece3f3b48389422dd56c6209b86a746ce81bfe570d850631d0b24e5abd6757"} Jul 20 12:50:49.264486 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:49.264159 2570 scope.go:117] "RemoveContainer" containerID="31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b" Jul 20 12:50:49.273457 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:49.273433 2570 scope.go:117] "RemoveContainer" containerID="31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b" Jul 20 12:50:49.273744 ip-10-0-141-229 kubenswrapper[2570]: E0720 12:50:49.273726 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b\": container with ID starting with 31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b not found: ID does not exist" containerID="31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b" Jul 20 12:50:49.273797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:49.273755 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b"} err="failed to get container status \"31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b\": rpc error: code = NotFound desc = could not find container \"31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b\": container with ID starting with 31b32fa2a91e56b0dd34458a35397ce05c6acf9aacfd105b8138861b8c8b093b not found: ID does not exist" Jul 20 12:50:49.286215 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:49.286183 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-77zq7"] Jul 20 12:50:49.293416 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:49.293356 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-77zq7"] Jul 20 12:50:50.520728 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:50.520683 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa" path="/var/lib/kubelet/pods/4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa/volumes" Jul 20 12:50:55.385966 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:55.385916 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-t86cr" Jul 20 12:50:55.386420 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:55.385985 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-t86cr" Jul 20 12:50:55.408825 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:55.408799 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-t86cr" Jul 20 12:50:56.318316 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:50:56.318285 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-t86cr" Jul 20 12:51:14.166905 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:14.166864 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:16.774857 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:16.774818 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4"] Jul 20 12:51:16.775265 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:16.775138 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa" containerName="registry-server" Jul 20 12:51:16.775265 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:16.775150 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa" containerName="registry-server" Jul 20 12:51:16.775265 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:16.775210 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ea30ee3-42e6-4d7b-9300-3cdbc89b7aaa" containerName="registry-server" Jul 20 12:51:16.778189 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:16.778170 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4" Jul 20 12:51:16.780821 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:16.780794 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 20 12:51:16.781361 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:16.781339 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-sds82\"" Jul 20 12:51:16.790139 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:16.790110 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfmvd\" (UniqueName: \"kubernetes.io/projected/9cb81595-2f69-4f83-bfc2-2e3ff63d5223-kube-api-access-kfmvd\") pod \"dns-operator-controller-manager-864dd78c4c-qppk4\" (UID: \"9cb81595-2f69-4f83-bfc2-2e3ff63d5223\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4" Jul 20 12:51:16.793048 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:16.793019 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4"] Jul 20 12:51:16.891395 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:16.891331 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfmvd\" (UniqueName: \"kubernetes.io/projected/9cb81595-2f69-4f83-bfc2-2e3ff63d5223-kube-api-access-kfmvd\") pod \"dns-operator-controller-manager-864dd78c4c-qppk4\" (UID: \"9cb81595-2f69-4f83-bfc2-2e3ff63d5223\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4" Jul 20 12:51:16.906003 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:16.905976 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfmvd\" (UniqueName: \"kubernetes.io/projected/9cb81595-2f69-4f83-bfc2-2e3ff63d5223-kube-api-access-kfmvd\") pod \"dns-operator-controller-manager-864dd78c4c-qppk4\" (UID: \"9cb81595-2f69-4f83-bfc2-2e3ff63d5223\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4" Jul 20 12:51:17.089002 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:17.088954 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4" Jul 20 12:51:17.249699 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:17.249664 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4"] Jul 20 12:51:17.253887 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:51:17.253856 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb81595_2f69_4f83_bfc2_2e3ff63d5223.slice/crio-b241bb94cea2b75b2e267a4a8eac070e003ece6f393393abc4ae36c8f713d293 WatchSource:0}: Error finding container b241bb94cea2b75b2e267a4a8eac070e003ece6f393393abc4ae36c8f713d293: Status 404 returned error can't find the container with id b241bb94cea2b75b2e267a4a8eac070e003ece6f393393abc4ae36c8f713d293 Jul 20 12:51:17.386478 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:17.386361 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4" event={"ID":"9cb81595-2f69-4f83-bfc2-2e3ff63d5223","Type":"ContainerStarted","Data":"b241bb94cea2b75b2e267a4a8eac070e003ece6f393393abc4ae36c8f713d293"} Jul 20 12:51:17.522218 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:17.522176 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:18.314862 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:18.314796 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:18.836101 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:18.836055 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:18.931634 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:18.931589 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:19.030918 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:19.030880 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:19.132578 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:19.132462 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:19.306518 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:19.306498 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 20 12:51:19.399884 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:19.399790 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4" event={"ID":"9cb81595-2f69-4f83-bfc2-2e3ff63d5223","Type":"ContainerStarted","Data":"91195e210cbe3c0f4fcf190b4f864e5973a5f796e943c53eb216a61f7e5ba6a5"} Jul 20 12:51:19.399884 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:19.399875 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4" Jul 20 12:51:19.448888 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:19.448835 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4" podStartSLOduration=1.4043594640000001 podStartE2EDuration="3.448819898s" totalImagesPullingTime="2.044460434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:51:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:51:19.448469954 +0000 UTC m=+421.420675786" watchObservedRunningTime="2026-07-20 12:51:19.448819898 +0000 UTC m=+421.421025729" Jul 20 12:51:19.583256 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:19.583223 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:19.680022 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:19.679950 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:20.866448 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:20.866417 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp"] Jul 20 12:51:20.872994 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:20.872977 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp" Jul 20 12:51:20.875901 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:20.875879 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-h8dsg\"" Jul 20 12:51:20.883789 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:20.883768 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp"] Jul 20 12:51:20.919865 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:20.919832 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kf56\" (UniqueName: \"kubernetes.io/projected/66326b8e-f834-450e-8d10-13ace27f33ed-kube-api-access-2kf56\") pod \"authorino-operator-75cc8d5cb7-x9pvp\" (UID: \"66326b8e-f834-450e-8d10-13ace27f33ed\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp" Jul 20 12:51:21.020624 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:21.020587 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kf56\" (UniqueName: \"kubernetes.io/projected/66326b8e-f834-450e-8d10-13ace27f33ed-kube-api-access-2kf56\") pod \"authorino-operator-75cc8d5cb7-x9pvp\" (UID: \"66326b8e-f834-450e-8d10-13ace27f33ed\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp" Jul 20 12:51:21.033454 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:21.033420 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kf56\" (UniqueName: \"kubernetes.io/projected/66326b8e-f834-450e-8d10-13ace27f33ed-kube-api-access-2kf56\") pod \"authorino-operator-75cc8d5cb7-x9pvp\" (UID: \"66326b8e-f834-450e-8d10-13ace27f33ed\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp" Jul 20 12:51:21.182852 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:21.182747 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp" Jul 20 12:51:21.336999 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:21.336969 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp"] Jul 20 12:51:21.339634 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:51:21.339598 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66326b8e_f834_450e_8d10_13ace27f33ed.slice/crio-97cc0e8151fb08737eb05d6a4d2193ff350c723bf45f27c621db01736fb96253 WatchSource:0}: Error finding container 97cc0e8151fb08737eb05d6a4d2193ff350c723bf45f27c621db01736fb96253: Status 404 returned error can't find the container with id 97cc0e8151fb08737eb05d6a4d2193ff350c723bf45f27c621db01736fb96253 Jul 20 12:51:21.409119 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:21.409086 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp" event={"ID":"66326b8e-f834-450e-8d10-13ace27f33ed","Type":"ContainerStarted","Data":"97cc0e8151fb08737eb05d6a4d2193ff350c723bf45f27c621db01736fb96253"} Jul 20 12:51:21.936077 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:21.936038 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:22.254441 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:22.254334 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:24.424785 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:24.424749 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp" event={"ID":"66326b8e-f834-450e-8d10-13ace27f33ed","Type":"ContainerStarted","Data":"b584a4bdf9c1d644a98a564e84d91f66e4f1f962a7422acd03d299b936a58254"} Jul 20 12:51:24.425238 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:24.424875 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp" Jul 20 12:51:24.470535 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:24.470488 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:24.483162 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:24.483107 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp" podStartSLOduration=2.2711088139999998 podStartE2EDuration="4.483090021s" totalImagesPullingTime="2.211981207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:51:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:51:24.475837767 +0000 UTC m=+426.448043600" watchObservedRunningTime="2026-07-20 12:51:24.483090021 +0000 UTC m=+426.455295915" Jul 20 12:51:25.415114 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:25.415081 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:26.103899 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:26.103866 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:29.590394 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:29.590342 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:30.188866 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:30.188799 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:30.289767 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:30.289731 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:30.389318 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:30.389285 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:30.407130 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:30.407100 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qppk4" Jul 20 12:51:31.368004 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:31.367972 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:31.483411 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:31.483357 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:35.432044 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:35.432017 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x9pvp" Jul 20 12:51:36.922519 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:36.922486 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:37.001435 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:37.001394 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:37.334020 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:37.333987 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:37.436011 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:37.435980 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:39.731359 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:39.731320 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:39.838089 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:39.838054 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:40.007112 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.007027 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l"] Jul 20 12:51:40.010647 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.010615 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" Jul 20 12:51:40.012898 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.012870 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 20 12:51:40.014232 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.014206 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 20 12:51:40.014527 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.014506 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5htl2\"" Jul 20 12:51:40.020482 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.020455 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l"] Jul 20 12:51:40.020643 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.020622 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:40.083780 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.083736 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc0684d0-174c-4ec6-9bb4-f4947db32687-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7zn8l\" (UID: \"fc0684d0-174c-4ec6-9bb4-f4947db32687\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" Jul 20 12:51:40.083780 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.083783 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fc0684d0-174c-4ec6-9bb4-f4947db32687-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7zn8l\" (UID: \"fc0684d0-174c-4ec6-9bb4-f4947db32687\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" Jul 20 12:51:40.083999 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.083925 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lktm\" (UniqueName: \"kubernetes.io/projected/fc0684d0-174c-4ec6-9bb4-f4947db32687-kube-api-access-4lktm\") pod \"kuadrant-console-plugin-66b5774985-7zn8l\" (UID: \"fc0684d0-174c-4ec6-9bb4-f4947db32687\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" Jul 20 12:51:40.129604 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.129567 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:40.184545 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.184503 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc0684d0-174c-4ec6-9bb4-f4947db32687-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7zn8l\" (UID: \"fc0684d0-174c-4ec6-9bb4-f4947db32687\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" Jul 20 12:51:40.184689 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.184555 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fc0684d0-174c-4ec6-9bb4-f4947db32687-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7zn8l\" (UID: \"fc0684d0-174c-4ec6-9bb4-f4947db32687\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" Jul 20 12:51:40.184689 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.184642 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lktm\" (UniqueName: \"kubernetes.io/projected/fc0684d0-174c-4ec6-9bb4-f4947db32687-kube-api-access-4lktm\") pod \"kuadrant-console-plugin-66b5774985-7zn8l\" (UID: \"fc0684d0-174c-4ec6-9bb4-f4947db32687\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" Jul 20 12:51:40.185245 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.185224 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fc0684d0-174c-4ec6-9bb4-f4947db32687-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7zn8l\" (UID: \"fc0684d0-174c-4ec6-9bb4-f4947db32687\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" Jul 20 12:51:40.187005 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.186980 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc0684d0-174c-4ec6-9bb4-f4947db32687-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7zn8l\" (UID: \"fc0684d0-174c-4ec6-9bb4-f4947db32687\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" Jul 20 12:51:40.193242 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.193215 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lktm\" (UniqueName: \"kubernetes.io/projected/fc0684d0-174c-4ec6-9bb4-f4947db32687-kube-api-access-4lktm\") pod \"kuadrant-console-plugin-66b5774985-7zn8l\" (UID: \"fc0684d0-174c-4ec6-9bb4-f4947db32687\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" Jul 20 12:51:40.321541 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.321429 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" Jul 20 12:51:40.463383 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.463306 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l"] Jul 20 12:51:40.467391 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:51:40.467347 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc0684d0_174c_4ec6_9bb4_f4947db32687.slice/crio-4c650257d61f19c6c2bd8bf6420248e2d8062f38d58090a2ce56e19542682008 WatchSource:0}: Error finding container 4c650257d61f19c6c2bd8bf6420248e2d8062f38d58090a2ce56e19542682008: Status 404 returned error can't find the container with id 4c650257d61f19c6c2bd8bf6420248e2d8062f38d58090a2ce56e19542682008 Jul 20 12:51:40.496417 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:40.496349 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" event={"ID":"fc0684d0-174c-4ec6-9bb4-f4947db32687","Type":"ContainerStarted","Data":"4c650257d61f19c6c2bd8bf6420248e2d8062f38d58090a2ce56e19542682008"} Jul 20 12:51:43.296748 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:43.296710 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:43.361777 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:43.361725 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:50.202447 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:50.202347 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:50.339212 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:50.339169 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:52.843118 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:52.843078 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt"] Jul 20 12:51:52.941807 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:52.941759 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt"] Jul 20 12:51:52.942009 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:52.941871 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 12:51:52.944990 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:52.944959 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-krktl\"" Jul 20 12:51:53.112386 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:53.112276 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pv2gt\" (UID: \"99d932fd-a357-44fe-b2e3-1f7e5d410b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 12:51:53.112549 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:53.112386 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbjcz\" (UniqueName: \"kubernetes.io/projected/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-kube-api-access-jbjcz\") pod \"kuadrant-operator-controller-manager-86878c769f-pv2gt\" (UID: \"99d932fd-a357-44fe-b2e3-1f7e5d410b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 12:51:53.213670 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:53.213630 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbjcz\" (UniqueName: \"kubernetes.io/projected/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-kube-api-access-jbjcz\") pod \"kuadrant-operator-controller-manager-86878c769f-pv2gt\" (UID: \"99d932fd-a357-44fe-b2e3-1f7e5d410b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 12:51:53.213923 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:53.213809 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pv2gt\" (UID: \"99d932fd-a357-44fe-b2e3-1f7e5d410b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 12:51:53.214189 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:53.214171 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pv2gt\" (UID: \"99d932fd-a357-44fe-b2e3-1f7e5d410b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 12:51:53.225951 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:53.225917 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbjcz\" (UniqueName: \"kubernetes.io/projected/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-kube-api-access-jbjcz\") pod \"kuadrant-operator-controller-manager-86878c769f-pv2gt\" (UID: \"99d932fd-a357-44fe-b2e3-1f7e5d410b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 12:51:53.254638 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:53.254592 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 12:51:55.195207 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:55.195170 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:55.304000 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:55.303961 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:55.410967 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:55.410922 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:55.514422 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:55.514298 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:57.772730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:57.772655 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:51:57.866483 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:51:57.866448 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:01.364595 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:01.364560 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:01.465420 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:01.465386 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:01.815042 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:01.815014 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt"] Jul 20 12:52:01.826888 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:52:01.826858 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d932fd_a357_44fe_b2e3_1f7e5d410b4d.slice/crio-bc88cfa44301d89edff67139c46345573b8976f60084f4714eda74e2cf7a0e97 WatchSource:0}: Error finding container bc88cfa44301d89edff67139c46345573b8976f60084f4714eda74e2cf7a0e97: Status 404 returned error can't find the container with id bc88cfa44301d89edff67139c46345573b8976f60084f4714eda74e2cf7a0e97 Jul 20 12:52:02.609603 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:02.609549 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" event={"ID":"99d932fd-a357-44fe-b2e3-1f7e5d410b4d","Type":"ContainerStarted","Data":"bc88cfa44301d89edff67139c46345573b8976f60084f4714eda74e2cf7a0e97"} Jul 20 12:52:02.611318 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:02.611288 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" event={"ID":"fc0684d0-174c-4ec6-9bb4-f4947db32687","Type":"ContainerStarted","Data":"15fadd6faec495b27e0410d7a06d5ad0cd146a5fe81be145de7f79dd922fdf2c"} Jul 20 12:52:02.634096 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:02.629931 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7zn8l" podStartSLOduration=2.321184679 podStartE2EDuration="23.629908686s" totalImagesPullingTime="21.308724007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:52:02.627639009 +0000 UTC m=+464.599844836" watchObservedRunningTime="2026-07-20 12:52:02.629908686 +0000 UTC m=+464.602114521" Jul 20 12:52:06.631969 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:06.631922 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" event={"ID":"99d932fd-a357-44fe-b2e3-1f7e5d410b4d","Type":"ContainerStarted","Data":"5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876"} Jul 20 12:52:06.632391 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:06.632006 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 12:52:06.653088 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:06.653032 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" podStartSLOduration=10.741954698 podStartE2EDuration="14.653010891s" totalImagesPullingTime="3.911056193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:52:06.651199762 +0000 UTC m=+468.623405594" watchObservedRunningTime="2026-07-20 12:52:06.653010891 +0000 UTC m=+468.625216728" Jul 20 12:52:17.638642 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:17.638602 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 12:52:23.707681 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:23.707649 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:23.778712 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:23.778644 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:34.121289 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:34.121235 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:34.327563 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:34.327528 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:38.451210 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:38.451172 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:38.562084 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:38.562046 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:38.625099 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:38.625061 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:38.816401 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:38.816278 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:52:38.837977 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:38.837936 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:52:38.838181 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:38.838111 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" Jul 20 12:52:38.842740 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:38.840974 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 20 12:52:38.850449 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:38.850322 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:52:38.932216 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:38.932173 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b46vw\" (UniqueName: \"kubernetes.io/projected/e394caee-8e24-427a-b423-0d9fa708ea22-kube-api-access-b46vw\") pod \"limitador-limitador-54d6d988d5-5cvxg\" (UID: \"e394caee-8e24-427a-b423-0d9fa708ea22\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" Jul 20 12:52:38.932428 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:38.932259 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e394caee-8e24-427a-b423-0d9fa708ea22-config-file\") pod \"limitador-limitador-54d6d988d5-5cvxg\" (UID: \"e394caee-8e24-427a-b423-0d9fa708ea22\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" Jul 20 12:52:39.033118 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:39.033078 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b46vw\" (UniqueName: \"kubernetes.io/projected/e394caee-8e24-427a-b423-0d9fa708ea22-kube-api-access-b46vw\") pod \"limitador-limitador-54d6d988d5-5cvxg\" (UID: \"e394caee-8e24-427a-b423-0d9fa708ea22\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" Jul 20 12:52:39.033339 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:39.033175 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e394caee-8e24-427a-b423-0d9fa708ea22-config-file\") pod \"limitador-limitador-54d6d988d5-5cvxg\" (UID: \"e394caee-8e24-427a-b423-0d9fa708ea22\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" Jul 20 12:52:39.033851 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:39.033831 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e394caee-8e24-427a-b423-0d9fa708ea22-config-file\") pod \"limitador-limitador-54d6d988d5-5cvxg\" (UID: \"e394caee-8e24-427a-b423-0d9fa708ea22\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" Jul 20 12:52:39.042487 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:39.042406 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b46vw\" (UniqueName: \"kubernetes.io/projected/e394caee-8e24-427a-b423-0d9fa708ea22-kube-api-access-b46vw\") pod \"limitador-limitador-54d6d988d5-5cvxg\" (UID: \"e394caee-8e24-427a-b423-0d9fa708ea22\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" Jul 20 12:52:39.155761 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:39.155719 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" Jul 20 12:52:39.305214 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:39.305071 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:52:39.307880 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:52:39.307840 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode394caee_8e24_427a_b423_0d9fa708ea22.slice/crio-20ead04d65106543304097f5006fc0b83229fb10e45d33de8e517e93d946c94f WatchSource:0}: Error finding container 20ead04d65106543304097f5006fc0b83229fb10e45d33de8e517e93d946c94f: Status 404 returned error can't find the container with id 20ead04d65106543304097f5006fc0b83229fb10e45d33de8e517e93d946c94f Jul 20 12:52:39.327859 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:39.327818 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:39.428275 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:39.428169 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:39.789656 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:39.789562 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" event={"ID":"e394caee-8e24-427a-b423-0d9fa708ea22","Type":"ContainerStarted","Data":"20ead04d65106543304097f5006fc0b83229fb10e45d33de8e517e93d946c94f"} Jul 20 12:52:42.806498 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:42.806457 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" event={"ID":"e394caee-8e24-427a-b423-0d9fa708ea22","Type":"ContainerStarted","Data":"fb030b507ff104a7fb54ea19cef15166d9f5e5afeb2d170bfe4af5c36c172e0f"} Jul 20 12:52:42.807004 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:42.806588 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" Jul 20 12:52:42.822383 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:42.822303 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" podStartSLOduration=2.3152477830000002 podStartE2EDuration="4.822280593s" totalImagesPullingTime="2.50703281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:52:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:52:42.820490364 +0000 UTC m=+504.792696198" watchObservedRunningTime="2026-07-20 12:52:42.822280593 +0000 UTC m=+504.794486424" Jul 20 12:52:53.812869 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:53.812838 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5cvxg" Jul 20 12:52:59.688818 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:59.688780 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:52:59.784691 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:52:59.784646 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:00.219363 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:00.219331 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:08.208051 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:08.208013 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:08.308994 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:08.308951 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:12.835002 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:12.834919 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:12.935396 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:12.935348 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:13.189887 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:13.189795 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:20.283579 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:20.283544 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:20.350579 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:20.350542 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:20.421735 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:20.421705 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:20.869019 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:20.868988 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:23.209912 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:23.209875 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:24.082437 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:24.082398 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:24.258770 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:24.258734 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:43.718756 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:43.718722 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:43.792051 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:43.792022 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:55.886770 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:55.886734 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:55.986179 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:55.986143 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:56.466323 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:56.466287 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:56.819620 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:56.819536 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:57.055599 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:57.055550 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:57.185090 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:57.185058 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:53:57.228896 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:53:57.228863 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:54:09.039707 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:09.039674 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:54:09.176497 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:09.176461 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:54:18.478587 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:18.478560 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 12:54:18.487689 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:18.487661 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 12:54:20.766546 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:20.766491 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:54:21.309486 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:21.309444 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:54:25.269577 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:25.269541 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:54:27.277517 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:27.277467 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:54:28.025741 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:28.025693 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:54:43.109123 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:43.109042 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:54:44.223544 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:44.223503 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:54:44.722612 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:44.722577 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:54:51.109918 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:51.109882 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:54:52.705533 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:54:52.705497 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:55:01.083043 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:01.083004 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:55:01.907441 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:01.907405 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:55:02.009255 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:02.009217 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:55:03.700416 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:03.700379 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:55:12.940123 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:12.940087 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:55:13.376230 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.376196 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d"] Jul 20 12:55:13.379284 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.379261 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.382419 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.382393 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 12:55:13.382594 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.382467 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7sf77\"" Jul 20 12:55:13.383800 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.383778 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 20 12:55:13.383929 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.383816 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 12:55:13.392307 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.392274 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d"] Jul 20 12:55:13.410049 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.410014 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:55:13.517026 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.516976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.517026 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.517029 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.517327 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.517112 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.517327 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.517143 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k2qs\" (UniqueName: \"kubernetes.io/projected/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-kube-api-access-7k2qs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.517327 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.517186 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.517327 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.517235 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.618391 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.618341 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.618606 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.618405 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.618606 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.618445 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.618606 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.618470 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k2qs\" (UniqueName: \"kubernetes.io/projected/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-kube-api-access-7k2qs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.618764 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.618606 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.618764 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.618675 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.618920 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.618896 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.618968 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.618942 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.619047 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.619030 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.620730 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.620702 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.620901 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.620884 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.626914 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.626846 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k2qs\" (UniqueName: \"kubernetes.io/projected/34eb34d8-b3f3-4d8e-b0b8-698606098fb7-kube-api-access-7k2qs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kml9d\" (UID: \"34eb34d8-b3f3-4d8e-b0b8-698606098fb7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.689610 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.689556 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:13.829297 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.829264 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d"] Jul 20 12:55:13.835098 ip-10-0-141-229 kubenswrapper[2570]: W0720 12:55:13.832639 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34eb34d8_b3f3_4d8e_b0b8_698606098fb7.slice/crio-7137d6cf610cddad7e2d62e023769586f5aa917726e7f9f305f3abb9c59c3fa6 WatchSource:0}: Error finding container 7137d6cf610cddad7e2d62e023769586f5aa917726e7f9f305f3abb9c59c3fa6: Status 404 returned error can't find the container with id 7137d6cf610cddad7e2d62e023769586f5aa917726e7f9f305f3abb9c59c3fa6 Jul 20 12:55:13.836164 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:13.836139 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 12:55:14.502894 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:14.502842 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" event={"ID":"34eb34d8-b3f3-4d8e-b0b8-698606098fb7","Type":"ContainerStarted","Data":"7137d6cf610cddad7e2d62e023769586f5aa917726e7f9f305f3abb9c59c3fa6"} Jul 20 12:55:15.509232 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:15.509194 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" event={"ID":"34eb34d8-b3f3-4d8e-b0b8-698606098fb7","Type":"ContainerStarted","Data":"da5e22e162daddfff323ef226e923557bdd8b73b3930a653d5d18cd1ccec865a"} Jul 20 12:55:15.525798 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:15.525737 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" podStartSLOduration=1.433424901 podStartE2EDuration="2.525719523s" totalImagesPullingTime="1.092294622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 12:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 12:55:15.524950554 +0000 UTC m=+657.497156387" watchObservedRunningTime="2026-07-20 12:55:15.525719523 +0000 UTC m=+657.497925354" Jul 20 12:55:16.209804 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:16.209767 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:55:23.690822 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:23.690773 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:23.691455 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:23.690845 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:23.703860 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:23.703829 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:24.565798 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:24.565767 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kml9d" Jul 20 12:55:39.260486 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:39.260451 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:55:39.372394 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:39.372324 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:55:39.483468 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:39.483433 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:55:40.477436 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:40.477402 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:55:40.566049 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:55:40.566018 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:56:09.400498 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:56:09.400454 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:56:19.207796 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:56:19.207707 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:56:31.200435 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:56:31.200394 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:56:42.245094 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:56:42.245041 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:56:42.305189 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:56:42.305149 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:57:00.945055 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:57:00.945011 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:57:01.019082 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:57:01.019050 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:57:46.373882 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:57:46.373763 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:57:47.623900 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:57:47.623868 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:58:09.988873 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:09.988835 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:58:21.292998 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:21.292956 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:58:26.267255 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:26.267211 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:58:26.363795 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:26.363759 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:58:26.463299 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:26.463267 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:58:32.291672 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:32.291639 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:58:32.391562 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:32.391524 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:58:32.492870 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:32.492829 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:58:32.664293 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:32.664258 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:58:38.365897 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:38.365835 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:58:38.478996 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:38.478949 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:58:54.097652 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:58:54.097618 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:59:03.087404 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:59:03.086273 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:59:11.993105 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:59:11.993025 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:59:17.999723 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:59:17.999685 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:59:18.507502 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:59:18.507461 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 12:59:18.512473 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:59:18.512443 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 12:59:29.001797 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:59:29.001747 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:59:38.892880 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:59:38.892839 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 12:59:44.885150 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:59:44.885108 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 12:59:50.190001 ip-10-0-141-229 kubenswrapper[2570]: I0720 12:59:50.189961 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:00:00.211525 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:00:00.211484 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:00:22.100230 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:00:22.100190 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:00:34.092589 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:00:34.092549 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:00:56.859050 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:00:56.859014 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:01:10.463323 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:01:10.463288 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:01:12.606628 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:01:12.606589 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:01:24.997609 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:01:24.997572 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:01:34.599676 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:01:34.599636 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:01:44.601782 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:01:44.601739 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:01:52.666979 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:01:52.666940 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:01:52.767528 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:01:52.767494 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:01:54.595951 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:01:54.595913 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:01:55.630571 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:01:55.630539 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:02:04.092023 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:04.091983 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:02:29.290504 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:29.290418 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt"] Jul 20 13:02:29.291049 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:29.290621 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" podUID="99d932fd-a357-44fe-b2e3-1f7e5d410b4d" containerName="manager" containerID="cri-o://5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876" gracePeriod=10 Jul 20 13:02:32.156556 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.156531 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 13:02:32.290653 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.290555 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-extensions-socket-volume\") pod \"99d932fd-a357-44fe-b2e3-1f7e5d410b4d\" (UID: \"99d932fd-a357-44fe-b2e3-1f7e5d410b4d\") " Jul 20 13:02:32.290653 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.290633 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbjcz\" (UniqueName: \"kubernetes.io/projected/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-kube-api-access-jbjcz\") pod \"99d932fd-a357-44fe-b2e3-1f7e5d410b4d\" (UID: \"99d932fd-a357-44fe-b2e3-1f7e5d410b4d\") " Jul 20 13:02:32.291014 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.290990 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "99d932fd-a357-44fe-b2e3-1f7e5d410b4d" (UID: "99d932fd-a357-44fe-b2e3-1f7e5d410b4d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:02:32.292720 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.292697 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-kube-api-access-jbjcz" (OuterVolumeSpecName: "kube-api-access-jbjcz") pod "99d932fd-a357-44fe-b2e3-1f7e5d410b4d" (UID: "99d932fd-a357-44fe-b2e3-1f7e5d410b4d"). InnerVolumeSpecName "kube-api-access-jbjcz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:02:32.391562 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.391513 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-extensions-socket-volume\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:02:32.391562 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.391551 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbjcz\" (UniqueName: \"kubernetes.io/projected/99d932fd-a357-44fe-b2e3-1f7e5d410b4d-kube-api-access-jbjcz\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:02:32.549619 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.549523 2570 generic.go:356] "Generic (PLEG): container finished" podID="99d932fd-a357-44fe-b2e3-1f7e5d410b4d" containerID="5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876" exitCode=0 Jul 20 13:02:32.549619 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.549567 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" Jul 20 13:02:32.549619 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.549605 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" event={"ID":"99d932fd-a357-44fe-b2e3-1f7e5d410b4d","Type":"ContainerDied","Data":"5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876"} Jul 20 13:02:32.549907 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.549664 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt" event={"ID":"99d932fd-a357-44fe-b2e3-1f7e5d410b4d","Type":"ContainerDied","Data":"bc88cfa44301d89edff67139c46345573b8976f60084f4714eda74e2cf7a0e97"} Jul 20 13:02:32.549907 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.549692 2570 scope.go:117] "RemoveContainer" containerID="5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876" Jul 20 13:02:32.564841 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.564794 2570 scope.go:117] "RemoveContainer" containerID="5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876" Jul 20 13:02:32.565288 ip-10-0-141-229 kubenswrapper[2570]: E0720 13:02:32.565245 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876\": container with ID starting with 5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876 not found: ID does not exist" containerID="5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876" Jul 20 13:02:32.565416 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.565288 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876"} err="failed to get container status \"5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876\": rpc error: code = NotFound desc = could not find container \"5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876\": container with ID starting with 5720a60c7464d09de511890b5b3c44ec12240d5c1e7dce844f710246398ab876 not found: ID does not exist" Jul 20 13:02:32.566655 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.566634 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt"] Jul 20 13:02:32.570334 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:32.570307 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pv2gt"] Jul 20 13:02:34.521230 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:02:34.521191 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99d932fd-a357-44fe-b2e3-1f7e5d410b4d" path="/var/lib/kubelet/pods/99d932fd-a357-44fe-b2e3-1f7e5d410b4d/volumes" Jul 20 13:03:30.418081 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.418045 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb"] Jul 20 13:03:30.418518 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.418393 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d932fd-a357-44fe-b2e3-1f7e5d410b4d" containerName="manager" Jul 20 13:03:30.418518 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.418407 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d932fd-a357-44fe-b2e3-1f7e5d410b4d" containerName="manager" Jul 20 13:03:30.418518 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.418480 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d932fd-a357-44fe-b2e3-1f7e5d410b4d" containerName="manager" Jul 20 13:03:30.422593 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.422572 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:03:30.424795 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.424769 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-krktl\"" Jul 20 13:03:30.435076 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.435046 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb"] Jul 20 13:03:30.489243 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.489203 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpxpz\" (UniqueName: \"kubernetes.io/projected/2b40e4b5-77e1-47be-8d27-aa605ac2b87f-kube-api-access-rpxpz\") pod \"kuadrant-operator-controller-manager-86878c769f-l5dmb\" (UID: \"2b40e4b5-77e1-47be-8d27-aa605ac2b87f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:03:30.489454 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.489280 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b40e4b5-77e1-47be-8d27-aa605ac2b87f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l5dmb\" (UID: \"2b40e4b5-77e1-47be-8d27-aa605ac2b87f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:03:30.590544 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.590503 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpxpz\" (UniqueName: \"kubernetes.io/projected/2b40e4b5-77e1-47be-8d27-aa605ac2b87f-kube-api-access-rpxpz\") pod \"kuadrant-operator-controller-manager-86878c769f-l5dmb\" (UID: \"2b40e4b5-77e1-47be-8d27-aa605ac2b87f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:03:30.590725 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.590574 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b40e4b5-77e1-47be-8d27-aa605ac2b87f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l5dmb\" (UID: \"2b40e4b5-77e1-47be-8d27-aa605ac2b87f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:03:30.590951 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.590935 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b40e4b5-77e1-47be-8d27-aa605ac2b87f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l5dmb\" (UID: \"2b40e4b5-77e1-47be-8d27-aa605ac2b87f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:03:30.602826 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.602792 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpxpz\" (UniqueName: \"kubernetes.io/projected/2b40e4b5-77e1-47be-8d27-aa605ac2b87f-kube-api-access-rpxpz\") pod \"kuadrant-operator-controller-manager-86878c769f-l5dmb\" (UID: \"2b40e4b5-77e1-47be-8d27-aa605ac2b87f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:03:30.733686 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.733596 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:03:30.884894 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:30.884858 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb"] Jul 20 13:03:30.889233 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:03:30.889202 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b40e4b5_77e1_47be_8d27_aa605ac2b87f.slice/crio-02eb137d609d07e8a0eb3c24a753849106e4745477b6daf180f3747337c96df4 WatchSource:0}: Error finding container 02eb137d609d07e8a0eb3c24a753849106e4745477b6daf180f3747337c96df4: Status 404 returned error can't find the container with id 02eb137d609d07e8a0eb3c24a753849106e4745477b6daf180f3747337c96df4 Jul 20 13:03:31.820441 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:31.820400 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" event={"ID":"2b40e4b5-77e1-47be-8d27-aa605ac2b87f","Type":"ContainerStarted","Data":"237dbc4541a115949e9de70faf2424fb7e686031ca015298e8c1ebbf00fcc712"} Jul 20 13:03:31.820999 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:31.820454 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" event={"ID":"2b40e4b5-77e1-47be-8d27-aa605ac2b87f","Type":"ContainerStarted","Data":"02eb137d609d07e8a0eb3c24a753849106e4745477b6daf180f3747337c96df4"} Jul 20 13:03:31.820999 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:31.820481 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:03:31.839027 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:31.838961 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" podStartSLOduration=1.838935746 podStartE2EDuration="1.838935746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:03:31.837151807 +0000 UTC m=+1153.809357640" watchObservedRunningTime="2026-07-20 13:03:31.838935746 +0000 UTC m=+1153.811141580" Jul 20 13:03:42.828747 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:42.828711 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:03:50.492469 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:03:50.492426 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:04:02.193782 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:04:02.193735 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:04:12.900870 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:04:12.900829 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:04:18.535729 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:04:18.535696 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:04:18.539206 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:04:18.539177 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:04:26.988753 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:04:26.988716 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:04:37.002510 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:04:37.002472 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:04:46.891395 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:04:46.891342 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:04:56.391882 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:04:56.391840 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:05:07.502556 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:05:07.502516 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:05:18.491122 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:05:18.491033 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:05:27.298150 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:05:27.298109 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:05:41.693590 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:05:41.693547 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:06:02.207026 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:02.206987 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:06:10.011541 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:10.011500 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:06:17.933361 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:17.933322 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5"] Jul 20 13:06:17.936684 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:17.936662 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:17.939851 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:17.939822 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-cf09fe97-kserve-self-signed-certs\"" Jul 20 13:06:17.969866 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:17.969831 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5"] Jul 20 13:06:18.013152 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.013119 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:06:18.078939 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.078894 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tls-certs\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.078939 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.078941 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-dshm\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.079155 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.079030 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d557z\" (UniqueName: \"kubernetes.io/projected/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-kube-api-access-d557z\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.079155 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.079113 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tmp-dir\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.079155 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.079139 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-home\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.079254 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.079160 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-model-cache\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.180168 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.180119 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d557z\" (UniqueName: \"kubernetes.io/projected/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-kube-api-access-d557z\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.180328 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.180203 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tmp-dir\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.180328 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.180229 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-home\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.180328 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.180253 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-model-cache\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.180328 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.180316 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tls-certs\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.180533 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.180349 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-dshm\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.180682 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.180660 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tmp-dir\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.180745 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.180721 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-home\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.180800 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.180740 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-model-cache\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.182805 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.182783 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-dshm\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.182949 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.182929 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tls-certs\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.191275 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.191210 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d557z\" (UniqueName: \"kubernetes.io/projected/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-kube-api-access-d557z\") pod \"e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.246131 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.246092 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:18.389619 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.389580 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5"] Jul 20 13:06:18.393219 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:06:18.393185 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a50af0a_00b8_4fd2_b77a_7baa08cb7c93.slice/crio-8217f31abd32ec6e2bfed4e9742a2cfcc60d19edd122d89d1c08374b8a9f3a70 WatchSource:0}: Error finding container 8217f31abd32ec6e2bfed4e9742a2cfcc60d19edd122d89d1c08374b8a9f3a70: Status 404 returned error can't find the container with id 8217f31abd32ec6e2bfed4e9742a2cfcc60d19edd122d89d1c08374b8a9f3a70 Jul 20 13:06:18.599215 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.599180 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" event={"ID":"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93","Type":"ContainerStarted","Data":"c28b8039b0c6faf36e6374575a5adc9c80f8fdb121b9f44c566d1f4d0b617eed"} Jul 20 13:06:18.599386 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.599224 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" event={"ID":"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93","Type":"ContainerStarted","Data":"8217f31abd32ec6e2bfed4e9742a2cfcc60d19edd122d89d1c08374b8a9f3a70"} Jul 20 13:06:18.618744 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:18.618690 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" podStartSLOduration=1.6186729739999999 podStartE2EDuration="1.618672974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:06:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:06:18.617785439 +0000 UTC m=+1320.589991271" watchObservedRunningTime="2026-07-20 13:06:18.618672974 +0000 UTC m=+1320.590878860" Jul 20 13:06:20.707216 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:20.707177 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:06:27.677112 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.677073 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv"] Jul 20 13:06:27.680596 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.680573 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.682596 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.682573 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-cf09fe97-kserve-self-signed-certs\"" Jul 20 13:06:27.690940 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.690915 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv"] Jul 20 13:06:27.773334 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.773286 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-dshm\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.773574 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.773349 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-model-cache\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.773574 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.773399 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb30d51b-d304-4203-9e12-6371873eb927-tls-certs\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.773574 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.773471 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-home\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.773574 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.773530 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8rl7\" (UniqueName: \"kubernetes.io/projected/fb30d51b-d304-4203-9e12-6371873eb927-kube-api-access-q8rl7\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.773574 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.773572 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-tmp-dir\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.861934 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.861896 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:06:27.875157 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.875118 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8rl7\" (UniqueName: \"kubernetes.io/projected/fb30d51b-d304-4203-9e12-6371873eb927-kube-api-access-q8rl7\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.875409 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.875182 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-tmp-dir\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.875409 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.875272 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-dshm\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.875409 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.875326 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-model-cache\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.875409 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.875348 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb30d51b-d304-4203-9e12-6371873eb927-tls-certs\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.875690 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.875475 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-home\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.875744 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.875685 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-model-cache\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.875797 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.875747 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-tmp-dir\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.875919 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.875894 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-home\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.878303 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.878273 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-dshm\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.878544 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.878524 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb30d51b-d304-4203-9e12-6371873eb927-tls-certs\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.884490 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.884462 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8rl7\" (UniqueName: \"kubernetes.io/projected/fb30d51b-d304-4203-9e12-6371873eb927-kube-api-access-q8rl7\") pod \"e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:27.990677 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:27.990589 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:28.129870 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:28.129822 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv"] Jul 20 13:06:28.134711 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:06:28.134672 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb30d51b_d304_4203_9e12_6371873eb927.slice/crio-46bc6390c9a8e850fe7678ed8d241f8159c93ad125f6f12100a8edba4722686a WatchSource:0}: Error finding container 46bc6390c9a8e850fe7678ed8d241f8159c93ad125f6f12100a8edba4722686a: Status 404 returned error can't find the container with id 46bc6390c9a8e850fe7678ed8d241f8159c93ad125f6f12100a8edba4722686a Jul 20 13:06:28.246515 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:28.246418 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:28.246515 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:28.246484 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:28.260268 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:28.260231 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:28.650872 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:28.650814 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" event={"ID":"fb30d51b-d304-4203-9e12-6371873eb927","Type":"ContainerStarted","Data":"02b301383be7e925c596ac6029a6fddcd1ce28e9a0a2f3d39330a44696b0b41a"} Jul 20 13:06:28.650872 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:28.650873 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" event={"ID":"fb30d51b-d304-4203-9e12-6371873eb927","Type":"ContainerStarted","Data":"46bc6390c9a8e850fe7678ed8d241f8159c93ad125f6f12100a8edba4722686a"} Jul 20 13:06:28.667122 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:28.666896 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:06:28.667959 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:28.667916 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" podStartSLOduration=1.667900463 podStartE2EDuration="1.667900463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:06:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:06:28.666801131 +0000 UTC m=+1330.639006952" watchObservedRunningTime="2026-07-20 13:06:28.667900463 +0000 UTC m=+1330.640106295" Jul 20 13:06:37.991030 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:37.990985 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:37.991030 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:37.991041 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:38.003925 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:38.003893 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:38.710783 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:38.710741 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:06:40.509005 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:40.508914 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:06:56.636388 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:56.636338 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5"] Jul 20 13:06:56.662885 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:56.662855 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:06:57.626648 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:57.626613 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv"] Jul 20 13:06:57.660834 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:57.660779 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:06:58.897312 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:06:58.897271 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:07:11.657470 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:11.657426 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" podUID="1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" containerName="main" containerID="cri-o://c28b8039b0c6faf36e6374575a5adc9c80f8fdb121b9f44c566d1f4d0b617eed" gracePeriod=45 Jul 20 13:07:11.858642 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:11.858602 2570 generic.go:356] "Generic (PLEG): container finished" podID="1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" containerID="c28b8039b0c6faf36e6374575a5adc9c80f8fdb121b9f44c566d1f4d0b617eed" exitCode=0 Jul 20 13:07:11.858811 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:11.858681 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" event={"ID":"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93","Type":"ContainerDied","Data":"c28b8039b0c6faf36e6374575a5adc9c80f8fdb121b9f44c566d1f4d0b617eed"} Jul 20 13:07:11.915544 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:11.915472 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:07:12.083765 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.083727 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-dshm\") pod \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " Jul 20 13:07:12.083977 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.083803 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-model-cache\") pod \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " Jul 20 13:07:12.083977 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.083831 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tmp-dir\") pod \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " Jul 20 13:07:12.083977 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.083877 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tls-certs\") pod \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " Jul 20 13:07:12.083977 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.083902 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d557z\" (UniqueName: \"kubernetes.io/projected/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-kube-api-access-d557z\") pod \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " Jul 20 13:07:12.083977 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.083930 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-home\") pod \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\" (UID: \"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93\") " Jul 20 13:07:12.084253 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.084076 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-model-cache" (OuterVolumeSpecName: "model-cache") pod "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" (UID: "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:07:12.084253 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.084102 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" (UID: "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:07:12.084253 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.084210 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-home" (OuterVolumeSpecName: "home") pod "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" (UID: "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:07:12.084456 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.084312 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:12.084456 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.084323 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:12.084456 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.084331 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:12.086199 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.086170 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-kube-api-access-d557z" (OuterVolumeSpecName: "kube-api-access-d557z") pod "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" (UID: "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93"). InnerVolumeSpecName "kube-api-access-d557z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:07:12.086424 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.086401 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" (UID: "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:07:12.086503 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.086419 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-dshm" (OuterVolumeSpecName: "dshm") pod "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" (UID: "1a50af0a-00b8-4fd2-b77a-7baa08cb7c93"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:07:12.185846 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.185748 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:12.185846 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.185783 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d557z\" (UniqueName: \"kubernetes.io/projected/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-kube-api-access-d557z\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:12.185846 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.185794 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:12.651019 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.650946 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" podUID="fb30d51b-d304-4203-9e12-6371873eb927" containerName="main" containerID="cri-o://02b301383be7e925c596ac6029a6fddcd1ce28e9a0a2f3d39330a44696b0b41a" gracePeriod=45 Jul 20 13:07:12.864970 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.864936 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" Jul 20 13:07:12.864970 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.864956 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5" event={"ID":"1a50af0a-00b8-4fd2-b77a-7baa08cb7c93","Type":"ContainerDied","Data":"8217f31abd32ec6e2bfed4e9742a2cfcc60d19edd122d89d1c08374b8a9f3a70"} Jul 20 13:07:12.865511 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.865016 2570 scope.go:117] "RemoveContainer" containerID="c28b8039b0c6faf36e6374575a5adc9c80f8fdb121b9f44c566d1f4d0b617eed" Jul 20 13:07:12.866789 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.866758 2570 generic.go:356] "Generic (PLEG): container finished" podID="fb30d51b-d304-4203-9e12-6371873eb927" containerID="02b301383be7e925c596ac6029a6fddcd1ce28e9a0a2f3d39330a44696b0b41a" exitCode=0 Jul 20 13:07:12.866910 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.866840 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" event={"ID":"fb30d51b-d304-4203-9e12-6371873eb927","Type":"ContainerDied","Data":"02b301383be7e925c596ac6029a6fddcd1ce28e9a0a2f3d39330a44696b0b41a"} Jul 20 13:07:12.880712 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.880663 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5"] Jul 20 13:07:12.883453 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.883421 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-cf09fe97-kserve-75d59b8579-z2vh5"] Jul 20 13:07:12.914246 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:12.914218 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:07:13.095394 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.095338 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-home\") pod \"fb30d51b-d304-4203-9e12-6371873eb927\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " Jul 20 13:07:13.095561 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.095455 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-model-cache\") pod \"fb30d51b-d304-4203-9e12-6371873eb927\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " Jul 20 13:07:13.095561 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.095476 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8rl7\" (UniqueName: \"kubernetes.io/projected/fb30d51b-d304-4203-9e12-6371873eb927-kube-api-access-q8rl7\") pod \"fb30d51b-d304-4203-9e12-6371873eb927\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " Jul 20 13:07:13.095561 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.095496 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb30d51b-d304-4203-9e12-6371873eb927-tls-certs\") pod \"fb30d51b-d304-4203-9e12-6371873eb927\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " Jul 20 13:07:13.095561 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.095529 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-tmp-dir\") pod \"fb30d51b-d304-4203-9e12-6371873eb927\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " Jul 20 13:07:13.095737 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.095563 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-dshm\") pod \"fb30d51b-d304-4203-9e12-6371873eb927\" (UID: \"fb30d51b-d304-4203-9e12-6371873eb927\") " Jul 20 13:07:13.095737 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.095642 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-home" (OuterVolumeSpecName: "home") pod "fb30d51b-d304-4203-9e12-6371873eb927" (UID: "fb30d51b-d304-4203-9e12-6371873eb927"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:07:13.095800 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.095750 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-model-cache" (OuterVolumeSpecName: "model-cache") pod "fb30d51b-d304-4203-9e12-6371873eb927" (UID: "fb30d51b-d304-4203-9e12-6371873eb927"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:07:13.095849 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.095826 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fb30d51b-d304-4203-9e12-6371873eb927" (UID: "fb30d51b-d304-4203-9e12-6371873eb927"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:07:13.095887 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.095856 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:13.095887 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.095874 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:13.097705 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.097679 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-dshm" (OuterVolumeSpecName: "dshm") pod "fb30d51b-d304-4203-9e12-6371873eb927" (UID: "fb30d51b-d304-4203-9e12-6371873eb927"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:07:13.097811 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.097717 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb30d51b-d304-4203-9e12-6371873eb927-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fb30d51b-d304-4203-9e12-6371873eb927" (UID: "fb30d51b-d304-4203-9e12-6371873eb927"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:07:13.097852 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.097808 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb30d51b-d304-4203-9e12-6371873eb927-kube-api-access-q8rl7" (OuterVolumeSpecName: "kube-api-access-q8rl7") pod "fb30d51b-d304-4203-9e12-6371873eb927" (UID: "fb30d51b-d304-4203-9e12-6371873eb927"). InnerVolumeSpecName "kube-api-access-q8rl7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:07:13.197080 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.197036 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8rl7\" (UniqueName: \"kubernetes.io/projected/fb30d51b-d304-4203-9e12-6371873eb927-kube-api-access-q8rl7\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:13.197080 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.197075 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb30d51b-d304-4203-9e12-6371873eb927-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:13.197317 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.197108 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:13.197317 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.197116 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb30d51b-d304-4203-9e12-6371873eb927-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:07:13.873301 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.873270 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" Jul 20 13:07:13.873801 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.873270 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv" event={"ID":"fb30d51b-d304-4203-9e12-6371873eb927","Type":"ContainerDied","Data":"46bc6390c9a8e850fe7678ed8d241f8159c93ad125f6f12100a8edba4722686a"} Jul 20 13:07:13.873801 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.873414 2570 scope.go:117] "RemoveContainer" containerID="02b301383be7e925c596ac6029a6fddcd1ce28e9a0a2f3d39330a44696b0b41a" Jul 20 13:07:13.895131 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.895100 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv"] Jul 20 13:07:13.899069 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:13.899044 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-cf09fe97-kserve-5dfc44db7f-rzlqv"] Jul 20 13:07:14.521340 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:14.521306 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" path="/var/lib/kubelet/pods/1a50af0a-00b8-4fd2-b77a-7baa08cb7c93/volumes" Jul 20 13:07:14.521741 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:14.521727 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb30d51b-d304-4203-9e12-6371873eb927" path="/var/lib/kubelet/pods/fb30d51b-d304-4203-9e12-6371873eb927/volumes" Jul 20 13:07:15.293792 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:15.293753 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:07:32.991961 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:32.991916 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:07:47.792946 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:47.792906 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:07:57.188991 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:07:57.188949 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:08:02.653970 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:08:02.653928 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:08:11.198787 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:08:11.198684 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:08:21.193288 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:08:21.193249 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:08:31.494067 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:08:31.494027 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:08:48.193879 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:08:48.193845 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:08:55.058603 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:08:55.058561 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:08:55.131024 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:08:55.130991 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:08:57.690678 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:08:57.690629 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:09:06.895780 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:09:06.895742 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:09:18.568013 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:09:18.567983 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:09:18.569694 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:09:18.569669 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:09:21.603797 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:09:21.603753 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:09:25.768176 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:09:25.768137 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:09:34.397812 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:09:34.397772 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:09:46.227768 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:09:46.227680 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:09:47.987041 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:09:47.986999 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:00.729134 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:00.729096 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:10:05.014770 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:05.014724 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:05.174023 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:05.173980 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:05.571789 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:05.571753 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:11.620424 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:11.620354 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:11.732178 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:11.732145 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:11.850354 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:11.850315 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:12.236781 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:12.236741 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:12.717118 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:12.717080 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:14.823778 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:14.823741 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:15.029275 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:15.029243 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:15.168563 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:15.168528 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:15.557038 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:15.556937 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:15.736175 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:15.736139 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:15.862126 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:15.862086 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:20.292090 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:20.292056 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:10:50.929672 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:50.929629 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:53.901069 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:53.901030 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:10:55.469530 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:55.469498 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:55.632224 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:55.632188 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:55.850513 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:55.850481 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:56.107313 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:56.107227 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:56.207521 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:56.207488 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:56.390227 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:56.390150 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:56.706886 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:56.706804 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:57.030338 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:57.030240 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:59.198113 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:59.198081 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:59.309700 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:59.309670 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:59.468071 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:59.467986 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:59.813603 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:59.813521 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:10:59.916655 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:10:59.916620 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:06.692085 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:06.692047 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:11:24.077744 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.077703 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h"] Jul 20 13:11:24.078290 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.078243 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb30d51b-d304-4203-9e12-6371873eb927" containerName="main" Jul 20 13:11:24.078290 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.078264 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb30d51b-d304-4203-9e12-6371873eb927" containerName="main" Jul 20 13:11:24.078290 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.078276 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" containerName="main" Jul 20 13:11:24.078290 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.078285 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" containerName="main" Jul 20 13:11:24.078604 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.078393 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb30d51b-d304-4203-9e12-6371873eb927" containerName="main" Jul 20 13:11:24.078604 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.078412 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a50af0a-00b8-4fd2-b77a-7baa08cb7c93" containerName="main" Jul 20 13:11:24.081448 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.081423 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" Jul 20 13:11:24.083541 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.083516 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 20 13:11:24.083663 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.083521 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4s4l\"" Jul 20 13:11:24.089074 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.088663 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h"] Jul 20 13:11:24.214829 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.214794 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c509502c-fd4a-4a04-aae0-7f7c872be68f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h\" (UID: \"c509502c-fd4a-4a04-aae0-7f7c872be68f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" Jul 20 13:11:24.315449 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.315401 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c509502c-fd4a-4a04-aae0-7f7c872be68f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h\" (UID: \"c509502c-fd4a-4a04-aae0-7f7c872be68f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" Jul 20 13:11:24.316082 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.316059 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c509502c-fd4a-4a04-aae0-7f7c872be68f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h\" (UID: \"c509502c-fd4a-4a04-aae0-7f7c872be68f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" Jul 20 13:11:24.393222 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.393178 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" Jul 20 13:11:24.532909 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.532864 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h"] Jul 20 13:11:24.535865 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:11:24.535834 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc509502c_fd4a_4a04_aae0_7f7c872be68f.slice/crio-841be090454926ab9003069fd4ba21dee6b2508ebbb1e6e82df5da2be9dc00b2 WatchSource:0}: Error finding container 841be090454926ab9003069fd4ba21dee6b2508ebbb1e6e82df5da2be9dc00b2: Status 404 returned error can't find the container with id 841be090454926ab9003069fd4ba21dee6b2508ebbb1e6e82df5da2be9dc00b2 Jul 20 13:11:24.537820 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:24.537796 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 13:11:25.079158 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:25.079123 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" event={"ID":"c509502c-fd4a-4a04-aae0-7f7c872be68f","Type":"ContainerStarted","Data":"841be090454926ab9003069fd4ba21dee6b2508ebbb1e6e82df5da2be9dc00b2"} Jul 20 13:11:28.097419 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:28.097307 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" event={"ID":"c509502c-fd4a-4a04-aae0-7f7c872be68f","Type":"ContainerStarted","Data":"d80ab603823aa804b02b31245c300cc80b5b0bd3170a3946ec1de9c44806d336"} Jul 20 13:11:28.113138 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:28.113076 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" podStartSLOduration=1.069799889 podStartE2EDuration="4.113059143s" totalImagesPullingTime="3.043259254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:11:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:11:28.11244365 +0000 UTC m=+1630.084649482" watchObservedRunningTime="2026-07-20 13:11:28.113059143 +0000 UTC m=+1630.085264974" Jul 20 13:11:29.103444 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:29.103403 2570 generic.go:356] "Generic (PLEG): container finished" podID="c509502c-fd4a-4a04-aae0-7f7c872be68f" containerID="d80ab603823aa804b02b31245c300cc80b5b0bd3170a3946ec1de9c44806d336" exitCode=0 Jul 20 13:11:29.103835 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:29.103490 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" event={"ID":"c509502c-fd4a-4a04-aae0-7f7c872be68f","Type":"ContainerDied","Data":"d80ab603823aa804b02b31245c300cc80b5b0bd3170a3946ec1de9c44806d336"} Jul 20 13:11:30.246986 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:30.246949 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" Jul 20 13:11:30.377085 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:30.376983 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c509502c-fd4a-4a04-aae0-7f7c872be68f-maas-api-service-ca\") pod \"c509502c-fd4a-4a04-aae0-7f7c872be68f\" (UID: \"c509502c-fd4a-4a04-aae0-7f7c872be68f\") " Jul 20 13:11:30.377357 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:30.377334 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c509502c-fd4a-4a04-aae0-7f7c872be68f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c509502c-fd4a-4a04-aae0-7f7c872be68f" (UID: "c509502c-fd4a-4a04-aae0-7f7c872be68f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:11:30.478103 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:30.478058 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c509502c-fd4a-4a04-aae0-7f7c872be68f-maas-api-service-ca\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:11:31.114382 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:31.114349 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" Jul 20 13:11:31.114568 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:31.114345 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h" event={"ID":"c509502c-fd4a-4a04-aae0-7f7c872be68f","Type":"ContainerDied","Data":"841be090454926ab9003069fd4ba21dee6b2508ebbb1e6e82df5da2be9dc00b2"} Jul 20 13:11:31.114568 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:31.114493 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="841be090454926ab9003069fd4ba21dee6b2508ebbb1e6e82df5da2be9dc00b2" Jul 20 13:11:34.696767 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:34.696724 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:37.503386 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:37.503328 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:11:39.161426 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:39.161389 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:39.161922 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:39.161627 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h"] Jul 20 13:11:39.163947 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:39.163924 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7257abbc-gsg9h"] Jul 20 13:11:39.351658 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:39.351626 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:39.591544 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:39.591513 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:39.842998 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:39.842926 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:40.144006 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.143913 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6"] Jul 20 13:11:40.144529 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.144502 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c509502c-fd4a-4a04-aae0-7f7c872be68f" containerName="revoke-keys" Jul 20 13:11:40.144529 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.144527 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c509502c-fd4a-4a04-aae0-7f7c872be68f" containerName="revoke-keys" Jul 20 13:11:40.144703 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.144604 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c509502c-fd4a-4a04-aae0-7f7c872be68f" containerName="revoke-keys" Jul 20 13:11:40.148330 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.148301 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.151608 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.151584 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-de7145c9-openshift-default-dockercfg-28bqv\"" Jul 20 13:11:40.159830 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.159802 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6"] Jul 20 13:11:40.166791 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.166716 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-istio-token\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.166791 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.166786 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-certs\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.167335 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.166821 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bccc22b8-f025-4395-849e-890a1372059e-istio-podinfo\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.167335 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.166836 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bccc22b8-f025-4395-849e-890a1372059e-istiod-ca-cert\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.167335 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.166860 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s5ml\" (UniqueName: \"kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-kube-api-access-7s5ml\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.167335 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.166883 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-envoy\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.175011 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.174928 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-socket\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.175260 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.175235 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-credential-socket\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.175342 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.175283 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-data\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.182346 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.182314 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:40.276166 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276118 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-socket\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276359 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276191 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-credential-socket\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276359 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276216 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-data\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276359 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276235 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-istio-token\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276359 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276255 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-certs\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276359 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276292 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bccc22b8-f025-4395-849e-890a1372059e-istio-podinfo\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276359 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276316 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bccc22b8-f025-4395-849e-890a1372059e-istiod-ca-cert\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276742 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276458 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s5ml\" (UniqueName: \"kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-kube-api-access-7s5ml\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276742 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276510 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-envoy\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276742 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276625 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-socket\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276742 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276688 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-credential-socket\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276742 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276744 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-certs\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.276979 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.276783 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-data\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.277547 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.277524 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bccc22b8-f025-4395-849e-890a1372059e-istiod-ca-cert\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.278774 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.278740 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-envoy\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.278923 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.278899 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bccc22b8-f025-4395-849e-890a1372059e-istio-podinfo\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.284621 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.284592 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-istio-token\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.284801 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.284780 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s5ml\" (UniqueName: \"kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-kube-api-access-7s5ml\") pod \"e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.358425 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.358393 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:40.460434 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.460333 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:40.523927 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.523888 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c509502c-fd4a-4a04-aae0-7f7c872be68f" path="/var/lib/kubelet/pods/c509502c-fd4a-4a04-aae0-7f7c872be68f/volumes" Jul 20 13:11:40.604685 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.604656 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6"] Jul 20 13:11:40.607478 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:11:40.607447 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbccc22b8_f025_4395_849e_890a1372059e.slice/crio-09f4e535becb7210b9534eb33ed82f29f895c140d184670914a4e9f12b74a5b7 WatchSource:0}: Error finding container 09f4e535becb7210b9534eb33ed82f29f895c140d184670914a4e9f12b74a5b7: Status 404 returned error can't find the container with id 09f4e535becb7210b9534eb33ed82f29f895c140d184670914a4e9f12b74a5b7 Jul 20 13:11:40.609964 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.609929 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:11:40.610052 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.610001 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:11:40.610052 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:40.610034 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:11:41.164772 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:41.164737 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" event={"ID":"bccc22b8-f025-4395-849e-890a1372059e","Type":"ContainerStarted","Data":"37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838"} Jul 20 13:11:41.164972 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:41.164791 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" event={"ID":"bccc22b8-f025-4395-849e-890a1372059e","Type":"ContainerStarted","Data":"09f4e535becb7210b9534eb33ed82f29f895c140d184670914a4e9f12b74a5b7"} Jul 20 13:11:41.186694 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:41.186628 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" podStartSLOduration=1.186608587 podStartE2EDuration="1.186608587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:11:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:11:41.18534555 +0000 UTC m=+1643.157551381" watchObservedRunningTime="2026-07-20 13:11:41.186608587 +0000 UTC m=+1643.158814421" Jul 20 13:11:41.460789 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:41.460685 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:41.466871 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:41.466842 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:41.800209 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:41.800114 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:42.170165 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:42.170131 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:42.171286 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:42.171266 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:42.532340 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:42.532257 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:42.628328 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:42.628299 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:42.694849 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:42.694810 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:43.060643 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:43.060614 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:43.295299 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:43.295255 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:43.398415 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:43.398351 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:11:49.092761 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:49.092715 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:11:57.171756 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:57.171716 2570 patch_prober.go:28] interesting pod/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 20 13:11:57.172171 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:57.171787 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 20 13:11:57.241434 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:57.241398 2570 generic.go:356] "Generic (PLEG): container finished" podID="bccc22b8-f025-4395-849e-890a1372059e" containerID="37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838" exitCode=0 Jul 20 13:11:57.241602 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:57.241481 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" event={"ID":"bccc22b8-f025-4395-849e-890a1372059e","Type":"ContainerDied","Data":"37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838"} Jul 20 13:11:57.242000 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:57.241984 2570 scope.go:117] "RemoveContainer" containerID="37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838" Jul 20 13:11:57.243057 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:57.243028 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:11:57.243122 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:57.243099 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:11:57.243166 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:57.243132 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:11:58.255853 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:58.255804 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" event={"ID":"bccc22b8-f025-4395-849e-890a1372059e","Type":"ContainerStarted","Data":"04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2"} Jul 20 13:11:58.461405 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:58.461348 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:11:58.461713 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:58.461673 2570 patch_prober.go:28] interesting pod/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 20 13:11:58.461844 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:58.461737 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 20 13:11:59.461785 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:59.461732 2570 patch_prober.go:28] interesting pod/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 20 13:11:59.462323 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:11:59.461813 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 20 13:12:00.461777 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:00.461731 2570 patch_prober.go:28] interesting pod/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 20 13:12:00.461994 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:00.461802 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 20 13:12:01.460997 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:01.460957 2570 patch_prober.go:28] interesting pod/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 20 13:12:01.461187 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:01.461016 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 20 13:12:02.465942 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:02.465911 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:12:02.466324 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:02.466237 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:12:02.467030 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:02.467009 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:12:19.075443 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:19.075392 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:22.006053 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:22.006011 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:12:23.436098 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:23.436056 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:24.593227 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:24.593186 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6"] Jul 20 13:12:24.593765 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:24.593426 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" containerID="cri-o://04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2" gracePeriod=30 Jul 20 13:12:24.619334 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:24.619303 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:24.843068 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:24.843032 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:25.060281 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:25.060192 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:25.375307 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:25.375271 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:25.508677 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:25.508648 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:27.410071 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:27.410036 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:27.513489 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:27.513454 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:27.588657 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:27.588621 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:28.004084 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:28.004050 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:28.148239 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:28.148202 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:28.248252 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:28.248223 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:12:29.856340 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.856264 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:12:29.937900 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.937845 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-istio-token\") pod \"bccc22b8-f025-4395-849e-890a1372059e\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " Jul 20 13:12:29.937900 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.937908 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-certs\") pod \"bccc22b8-f025-4395-849e-890a1372059e\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " Jul 20 13:12:29.938170 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.937935 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bccc22b8-f025-4395-849e-890a1372059e-istiod-ca-cert\") pod \"bccc22b8-f025-4395-849e-890a1372059e\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " Jul 20 13:12:29.938170 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.937956 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-envoy\") pod \"bccc22b8-f025-4395-849e-890a1372059e\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " Jul 20 13:12:29.938170 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.937985 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s5ml\" (UniqueName: \"kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-kube-api-access-7s5ml\") pod \"bccc22b8-f025-4395-849e-890a1372059e\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " Jul 20 13:12:29.938170 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.938003 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-data\") pod \"bccc22b8-f025-4395-849e-890a1372059e\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " Jul 20 13:12:29.938170 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.938049 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-socket\") pod \"bccc22b8-f025-4395-849e-890a1372059e\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " Jul 20 13:12:29.938170 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.938076 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bccc22b8-f025-4395-849e-890a1372059e-istio-podinfo\") pod \"bccc22b8-f025-4395-849e-890a1372059e\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " Jul 20 13:12:29.938170 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.938108 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-credential-socket\") pod \"bccc22b8-f025-4395-849e-890a1372059e\" (UID: \"bccc22b8-f025-4395-849e-890a1372059e\") " Jul 20 13:12:29.938559 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.938393 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bccc22b8-f025-4395-849e-890a1372059e" (UID: "bccc22b8-f025-4395-849e-890a1372059e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:12:29.938617 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.938599 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bccc22b8-f025-4395-849e-890a1372059e" (UID: "bccc22b8-f025-4395-849e-890a1372059e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:12:29.939298 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.939266 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-data" (OuterVolumeSpecName: "istio-data") pod "bccc22b8-f025-4395-849e-890a1372059e" (UID: "bccc22b8-f025-4395-849e-890a1372059e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:12:29.939651 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.939621 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bccc22b8-f025-4395-849e-890a1372059e" (UID: "bccc22b8-f025-4395-849e-890a1372059e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:12:29.939840 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.939806 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bccc22b8-f025-4395-849e-890a1372059e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bccc22b8-f025-4395-849e-890a1372059e" (UID: "bccc22b8-f025-4395-849e-890a1372059e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:12:29.940534 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.940501 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bccc22b8-f025-4395-849e-890a1372059e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bccc22b8-f025-4395-849e-890a1372059e" (UID: "bccc22b8-f025-4395-849e-890a1372059e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 13:12:29.940626 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.940550 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-kube-api-access-7s5ml" (OuterVolumeSpecName: "kube-api-access-7s5ml") pod "bccc22b8-f025-4395-849e-890a1372059e" (UID: "bccc22b8-f025-4395-849e-890a1372059e"). InnerVolumeSpecName "kube-api-access-7s5ml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:12:29.940626 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.940568 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-istio-token" (OuterVolumeSpecName: "istio-token") pod "bccc22b8-f025-4395-849e-890a1372059e" (UID: "bccc22b8-f025-4395-849e-890a1372059e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:12:29.941278 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:29.941257 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bccc22b8-f025-4395-849e-890a1372059e" (UID: "bccc22b8-f025-4395-849e-890a1372059e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:12:30.039096 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.039055 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-istio-token\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:12:30.039096 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.039089 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:12:30.039096 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.039101 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bccc22b8-f025-4395-849e-890a1372059e-istiod-ca-cert\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:12:30.039349 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.039111 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-envoy\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:12:30.039349 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.039123 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s5ml\" (UniqueName: \"kubernetes.io/projected/bccc22b8-f025-4395-849e-890a1372059e-kube-api-access-7s5ml\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:12:30.039349 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.039134 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-istio-data\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:12:30.039349 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.039145 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-workload-socket\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:12:30.039349 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.039155 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bccc22b8-f025-4395-849e-890a1372059e-istio-podinfo\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:12:30.039349 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.039168 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bccc22b8-f025-4395-849e-890a1372059e-credential-socket\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:12:30.415779 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.415747 2570 generic.go:356] "Generic (PLEG): container finished" podID="bccc22b8-f025-4395-849e-890a1372059e" containerID="04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2" exitCode=0 Jul 20 13:12:30.415974 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.415800 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" Jul 20 13:12:30.415974 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.415807 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" event={"ID":"bccc22b8-f025-4395-849e-890a1372059e","Type":"ContainerDied","Data":"04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2"} Jul 20 13:12:30.415974 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.415851 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6" event={"ID":"bccc22b8-f025-4395-849e-890a1372059e","Type":"ContainerDied","Data":"09f4e535becb7210b9534eb33ed82f29f895c140d184670914a4e9f12b74a5b7"} Jul 20 13:12:30.415974 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.415873 2570 scope.go:117] "RemoveContainer" containerID="04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2" Jul 20 13:12:30.425894 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.425873 2570 scope.go:117] "RemoveContainer" containerID="37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838" Jul 20 13:12:30.435186 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.435164 2570 scope.go:117] "RemoveContainer" containerID="04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2" Jul 20 13:12:30.435603 ip-10-0-141-229 kubenswrapper[2570]: E0720 13:12:30.435576 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2\": container with ID starting with 04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2 not found: ID does not exist" containerID="04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2" Jul 20 13:12:30.435691 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.435611 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2"} err="failed to get container status \"04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2\": rpc error: code = NotFound desc = could not find container \"04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2\": container with ID starting with 04096a9430e8fa6af65f6f2dc242dede5c85577f8e14a9576380fe9e99e85cc2 not found: ID does not exist" Jul 20 13:12:30.435691 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.435632 2570 scope.go:117] "RemoveContainer" containerID="37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838" Jul 20 13:12:30.435923 ip-10-0-141-229 kubenswrapper[2570]: E0720 13:12:30.435883 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838\": container with ID starting with 37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838 not found: ID does not exist" containerID="37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838" Jul 20 13:12:30.435923 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.435912 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838"} err="failed to get container status \"37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838\": rpc error: code = NotFound desc = could not find container \"37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838\": container with ID starting with 37383a08b5d205b0f06e7d817484a70833e4bebd5232ea2053d41f49c7dd7838 not found: ID does not exist" Jul 20 13:12:30.438630 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.438603 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6"] Jul 20 13:12:30.444109 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.444067 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-de7145c9-openshift-default-58457cc57f-vd2j6"] Jul 20 13:12:30.521447 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:30.521409 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bccc22b8-f025-4395-849e-890a1372059e" path="/var/lib/kubelet/pods/bccc22b8-f025-4395-849e-890a1372059e/volumes" Jul 20 13:12:34.890310 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:34.890273 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:12:52.250432 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.250335 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v"] Jul 20 13:12:52.250830 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.250687 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" Jul 20 13:12:52.250830 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.250698 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" Jul 20 13:12:52.250830 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.250780 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" Jul 20 13:12:52.250830 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.250788 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" Jul 20 13:12:52.252729 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.252710 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" Jul 20 13:12:52.254851 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.254820 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 20 13:12:52.254992 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.254874 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4s4l\"" Jul 20 13:12:52.262296 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.262252 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v"] Jul 20 13:12:52.345317 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.345279 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d47862-df49-4042-8e2d-7bb1526c129d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5545572a-mwv2v\" (UID: \"53d47862-df49-4042-8e2d-7bb1526c129d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" Jul 20 13:12:52.446547 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.446508 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d47862-df49-4042-8e2d-7bb1526c129d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5545572a-mwv2v\" (UID: \"53d47862-df49-4042-8e2d-7bb1526c129d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" Jul 20 13:12:52.447197 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.447169 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d47862-df49-4042-8e2d-7bb1526c129d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5545572a-mwv2v\" (UID: \"53d47862-df49-4042-8e2d-7bb1526c129d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" Jul 20 13:12:52.563922 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.563828 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" Jul 20 13:12:52.715487 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:52.715452 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v"] Jul 20 13:12:52.719153 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:12:52.719123 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53d47862_df49_4042_8e2d_7bb1526c129d.slice/crio-38c9d5bbde7df110182180d21864ef1cc757c61dcb409cc36140bb78292bfeb8 WatchSource:0}: Error finding container 38c9d5bbde7df110182180d21864ef1cc757c61dcb409cc36140bb78292bfeb8: Status 404 returned error can't find the container with id 38c9d5bbde7df110182180d21864ef1cc757c61dcb409cc36140bb78292bfeb8 Jul 20 13:12:53.535175 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:53.535138 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" event={"ID":"53d47862-df49-4042-8e2d-7bb1526c129d","Type":"ContainerStarted","Data":"f689d9c88b8eb78185f7f585179db428abd6ac384e480c179cfe4e3a15a38c86"} Jul 20 13:12:53.535590 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:53.535196 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" event={"ID":"53d47862-df49-4042-8e2d-7bb1526c129d","Type":"ContainerStarted","Data":"38c9d5bbde7df110182180d21864ef1cc757c61dcb409cc36140bb78292bfeb8"} Jul 20 13:12:53.554246 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:53.554189 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" podStartSLOduration=1.554171695 podStartE2EDuration="1.554171695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:12:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:12:53.552993937 +0000 UTC m=+1715.525199780" watchObservedRunningTime="2026-07-20 13:12:53.554171695 +0000 UTC m=+1715.526377526" Jul 20 13:12:54.541349 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:54.541314 2570 generic.go:356] "Generic (PLEG): container finished" podID="53d47862-df49-4042-8e2d-7bb1526c129d" containerID="f689d9c88b8eb78185f7f585179db428abd6ac384e480c179cfe4e3a15a38c86" exitCode=0 Jul 20 13:12:54.541783 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:54.541407 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" event={"ID":"53d47862-df49-4042-8e2d-7bb1526c129d","Type":"ContainerDied","Data":"f689d9c88b8eb78185f7f585179db428abd6ac384e480c179cfe4e3a15a38c86"} Jul 20 13:12:55.687350 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:55.687323 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" Jul 20 13:12:55.777597 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:55.777560 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d47862-df49-4042-8e2d-7bb1526c129d-maas-api-service-ca\") pod \"53d47862-df49-4042-8e2d-7bb1526c129d\" (UID: \"53d47862-df49-4042-8e2d-7bb1526c129d\") " Jul 20 13:12:55.777950 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:55.777927 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53d47862-df49-4042-8e2d-7bb1526c129d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "53d47862-df49-4042-8e2d-7bb1526c129d" (UID: "53d47862-df49-4042-8e2d-7bb1526c129d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:12:55.879101 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:55.879070 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d47862-df49-4042-8e2d-7bb1526c129d-maas-api-service-ca\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:12:56.558325 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:56.558296 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" Jul 20 13:12:56.558545 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:56.558299 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v" event={"ID":"53d47862-df49-4042-8e2d-7bb1526c129d","Type":"ContainerDied","Data":"38c9d5bbde7df110182180d21864ef1cc757c61dcb409cc36140bb78292bfeb8"} Jul 20 13:12:56.558545 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:12:56.558423 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38c9d5bbde7df110182180d21864ef1cc757c61dcb409cc36140bb78292bfeb8" Jul 20 13:13:02.760253 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:02.760217 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:05.698729 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:05.698689 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:13:07.329027 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:07.328992 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v"] Jul 20 13:13:07.331567 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:07.331540 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5545572a-mwv2v"] Jul 20 13:13:07.340422 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:07.340388 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:07.502702 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:07.502668 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:07.717831 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:07.717798 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:08.193005 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:08.192970 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:08.293263 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:08.293231 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:08.400146 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:08.400098 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:08.521335 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:08.521250 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53d47862-df49-4042-8e2d-7bb1526c129d" path="/var/lib/kubelet/pods/53d47862-df49-4042-8e2d-7bb1526c129d/volumes" Jul 20 13:13:08.921546 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:08.921507 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:09.022961 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:09.022927 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:09.848362 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:09.848322 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:15.972237 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:15.972199 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:16.074720 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:16.074687 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:16.181799 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:16.181766 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:16.273935 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:16.273843 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:16.749333 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:16.749298 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:42.698804 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:42.698756 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:49.274904 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:49.274866 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:49.385718 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:49.385679 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:13:49.526222 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:13:49.526131 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:05.429536 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:05.429503 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:11.264782 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:11.264610 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:11.362181 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:11.362136 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:11.459497 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:11.459442 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:18.596132 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:18.596097 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:14:18.596975 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:18.596955 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:14:21.960288 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:21.960249 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:22.741208 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:22.741171 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:29.786077 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:29.786038 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:29.951998 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:29.951960 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:30.072914 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:30.072827 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:33.499709 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:33.499661 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:14:36.021565 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:36.021526 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:36.117476 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:36.117435 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:36.218596 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:36.218558 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:37.840399 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:37.840327 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:43.882120 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:43.882081 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:43.955195 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:43.955158 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:44.196629 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:44.196527 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:14:44.575840 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:44.575754 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:50.428160 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:50.428123 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:50.491278 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:50.491246 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:50.668991 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:50.668959 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:51.011677 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:51.011630 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:51.212572 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:51.212540 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:53.789552 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:53.789520 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:53.890185 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:53.890151 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:54.025160 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:54.025128 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:54.508659 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:54.508622 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:54.609358 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:54.609323 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:54.713197 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:54.713165 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:56.298154 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:56.298126 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:14:58.178697 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:58.178653 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:58.824567 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:58.824526 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:58.924173 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:58.924141 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:14:59.262044 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:14:59.262011 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:00.129749 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.129704 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf"] Jul 20 13:15:00.130143 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.130123 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" Jul 20 13:15:00.130232 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.130145 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bccc22b8-f025-4395-849e-890a1372059e" containerName="istio-proxy" Jul 20 13:15:00.130232 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.130166 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d47862-df49-4042-8e2d-7bb1526c129d" containerName="revoke-keys" Jul 20 13:15:00.130232 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.130174 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d47862-df49-4042-8e2d-7bb1526c129d" containerName="revoke-keys" Jul 20 13:15:00.130480 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.130280 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="53d47862-df49-4042-8e2d-7bb1526c129d" containerName="revoke-keys" Jul 20 13:15:00.133590 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.133562 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf" Jul 20 13:15:00.136669 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.136643 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4s4l\"" Jul 20 13:15:00.137562 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.137539 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2"] Jul 20 13:15:00.141526 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.141164 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2" Jul 20 13:15:00.142912 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.142880 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf"] Jul 20 13:15:00.147737 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.147698 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf" Jul 20 13:15:00.147901 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.147879 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2"] Jul 20 13:15:00.157431 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.157403 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2" Jul 20 13:15:00.302857 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.302820 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf"] Jul 20 13:15:00.307450 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:15:00.307420 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d98d12_bb4f_4a26_b72a_8a8ff1cbce89.slice/crio-1e57965681aa6215d6b98875d74aef1d66321870b8105e1ef7888e6319ff3695 WatchSource:0}: Error finding container 1e57965681aa6215d6b98875d74aef1d66321870b8105e1ef7888e6319ff3695: Status 404 returned error can't find the container with id 1e57965681aa6215d6b98875d74aef1d66321870b8105e1ef7888e6319ff3695 Jul 20 13:15:00.325135 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:00.325071 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2"] Jul 20 13:15:00.328399 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:15:00.328340 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c9155cb_766c_42b5_a4d9_9c61d8fdee20.slice/crio-f1ba7411777c14bdeee9b13bfe9f907c2fa613b0ca5d1a7d3fbe25b4e1bf30a0 WatchSource:0}: Error finding container f1ba7411777c14bdeee9b13bfe9f907c2fa613b0ca5d1a7d3fbe25b4e1bf30a0: Status 404 returned error can't find the container with id f1ba7411777c14bdeee9b13bfe9f907c2fa613b0ca5d1a7d3fbe25b4e1bf30a0 Jul 20 13:15:01.149050 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:01.149013 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf" event={"ID":"b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89","Type":"ContainerStarted","Data":"1bedc54b954d20e6080eb88a7edcddca205325b5ea4dd6505d6567cb2ca8e7ee"} Jul 20 13:15:01.149261 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:01.149107 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf" event={"ID":"b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89","Type":"ContainerStarted","Data":"1e57965681aa6215d6b98875d74aef1d66321870b8105e1ef7888e6319ff3695"} Jul 20 13:15:01.150628 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:01.150603 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2" event={"ID":"5c9155cb-766c-42b5-a4d9-9c61d8fdee20","Type":"ContainerStarted","Data":"e1e1dcfe093c0d194541ef4fc469c43e15b44a03e0df6761368b7a14cd84cffa"} Jul 20 13:15:01.150737 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:01.150643 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2" event={"ID":"5c9155cb-766c-42b5-a4d9-9c61d8fdee20","Type":"ContainerStarted","Data":"f1ba7411777c14bdeee9b13bfe9f907c2fa613b0ca5d1a7d3fbe25b4e1bf30a0"} Jul 20 13:15:01.162940 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:01.162886 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf" podStartSLOduration=1.162868974 podStartE2EDuration="1.162868974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:15:01.162320764 +0000 UTC m=+1843.134526597" watchObservedRunningTime="2026-07-20 13:15:01.162868974 +0000 UTC m=+1843.135074783" Jul 20 13:15:01.176576 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:01.176472 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2" podStartSLOduration=1.176452901 podStartE2EDuration="1.176452901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:15:01.175461005 +0000 UTC m=+1843.147666861" watchObservedRunningTime="2026-07-20 13:15:01.176452901 +0000 UTC m=+1843.148658732" Jul 20 13:15:02.157119 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:02.157080 2570 generic.go:356] "Generic (PLEG): container finished" podID="b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89" containerID="1bedc54b954d20e6080eb88a7edcddca205325b5ea4dd6505d6567cb2ca8e7ee" exitCode=7 Jul 20 13:15:02.157577 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:02.157152 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf" event={"ID":"b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89","Type":"ContainerDied","Data":"1bedc54b954d20e6080eb88a7edcddca205325b5ea4dd6505d6567cb2ca8e7ee"} Jul 20 13:15:02.157577 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:02.157313 2570 scope.go:117] "RemoveContainer" containerID="1bedc54b954d20e6080eb88a7edcddca205325b5ea4dd6505d6567cb2ca8e7ee" Jul 20 13:15:02.158954 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:02.158932 2570 generic.go:356] "Generic (PLEG): container finished" podID="5c9155cb-766c-42b5-a4d9-9c61d8fdee20" containerID="e1e1dcfe093c0d194541ef4fc469c43e15b44a03e0df6761368b7a14cd84cffa" exitCode=0 Jul 20 13:15:02.159060 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:02.158997 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2" event={"ID":"5c9155cb-766c-42b5-a4d9-9c61d8fdee20","Type":"ContainerDied","Data":"e1e1dcfe093c0d194541ef4fc469c43e15b44a03e0df6761368b7a14cd84cffa"} Jul 20 13:15:03.165867 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:03.165834 2570 generic.go:356] "Generic (PLEG): container finished" podID="b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89" containerID="f343dd11e76eb2afb47a791c349f53e04a8dd16a5ab8931c4e099e7e2fb20a5c" exitCode=0 Jul 20 13:15:03.166260 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:03.165930 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf" event={"ID":"b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89","Type":"ContainerDied","Data":"f343dd11e76eb2afb47a791c349f53e04a8dd16a5ab8931c4e099e7e2fb20a5c"} Jul 20 13:15:03.166260 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:03.166000 2570 scope.go:117] "RemoveContainer" containerID="1bedc54b954d20e6080eb88a7edcddca205325b5ea4dd6505d6567cb2ca8e7ee" Jul 20 13:15:03.299925 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:03.299862 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2" Jul 20 13:15:04.173589 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:04.173558 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2" event={"ID":"5c9155cb-766c-42b5-a4d9-9c61d8fdee20","Type":"ContainerDied","Data":"f1ba7411777c14bdeee9b13bfe9f907c2fa613b0ca5d1a7d3fbe25b4e1bf30a0"} Jul 20 13:15:04.173966 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:04.173615 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1ba7411777c14bdeee9b13bfe9f907c2fa613b0ca5d1a7d3fbe25b4e1bf30a0" Jul 20 13:15:04.173966 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:04.173576 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742555-t7lf2" Jul 20 13:15:04.309326 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:04.309296 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf" Jul 20 13:15:05.161668 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.161631 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:05.180101 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.180071 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf" Jul 20 13:15:05.180639 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.180072 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf" event={"ID":"b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89","Type":"ContainerDied","Data":"1e57965681aa6215d6b98875d74aef1d66321870b8105e1ef7888e6319ff3695"} Jul 20 13:15:05.180639 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.180197 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e57965681aa6215d6b98875d74aef1d66321870b8105e1ef7888e6319ff3695" Jul 20 13:15:05.261211 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.261172 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:05.505411 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.505318 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd"] Jul 20 13:15:05.505686 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.505672 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9155cb-766c-42b5-a4d9-9c61d8fdee20" containerName="cleanup" Jul 20 13:15:05.505725 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.505687 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9155cb-766c-42b5-a4d9-9c61d8fdee20" containerName="cleanup" Jul 20 13:15:05.505725 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.505702 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89" containerName="cleanup" Jul 20 13:15:05.505725 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.505708 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89" containerName="cleanup" Jul 20 13:15:05.505725 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.505719 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89" containerName="cleanup" Jul 20 13:15:05.505725 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.505725 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89" containerName="cleanup" Jul 20 13:15:05.505927 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.505780 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c9155cb-766c-42b5-a4d9-9c61d8fdee20" containerName="cleanup" Jul 20 13:15:05.505927 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.505791 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89" containerName="cleanup" Jul 20 13:15:05.505927 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.505799 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89" containerName="cleanup" Jul 20 13:15:05.508647 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.508628 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd" Jul 20 13:15:05.510830 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.510808 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 20 13:15:05.510976 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.510809 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4s4l\"" Jul 20 13:15:05.517795 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.517725 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd"] Jul 20 13:15:05.639293 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.639232 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df3e72f-e3b0-4213-aa87-576cab406cf8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd\" (UID: \"3df3e72f-e3b0-4213-aa87-576cab406cf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd" Jul 20 13:15:05.740063 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.740011 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df3e72f-e3b0-4213-aa87-576cab406cf8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd\" (UID: \"3df3e72f-e3b0-4213-aa87-576cab406cf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd" Jul 20 13:15:05.740962 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.740921 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df3e72f-e3b0-4213-aa87-576cab406cf8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd\" (UID: \"3df3e72f-e3b0-4213-aa87-576cab406cf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd" Jul 20 13:15:05.820528 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.820427 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd" Jul 20 13:15:05.962843 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:05.962809 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd"] Jul 20 13:15:05.965340 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:15:05.965298 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df3e72f_e3b0_4213_aa87_576cab406cf8.slice/crio-0a155b8732b14347bc095187e46446e10940aabf3cdaf6c9f71e822e7a790a16 WatchSource:0}: Error finding container 0a155b8732b14347bc095187e46446e10940aabf3cdaf6c9f71e822e7a790a16: Status 404 returned error can't find the container with id 0a155b8732b14347bc095187e46446e10940aabf3cdaf6c9f71e822e7a790a16 Jul 20 13:15:06.186887 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:06.186847 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd" event={"ID":"3df3e72f-e3b0-4213-aa87-576cab406cf8","Type":"ContainerStarted","Data":"0a155b8732b14347bc095187e46446e10940aabf3cdaf6c9f71e822e7a790a16"} Jul 20 13:15:07.193738 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:07.193639 2570 generic.go:356] "Generic (PLEG): container finished" podID="3df3e72f-e3b0-4213-aa87-576cab406cf8" containerID="0df178f88642a99657850e8ad9e463152c07f593aebec8e8aeab444245bde49d" exitCode=0 Jul 20 13:15:07.193738 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:07.193724 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd" event={"ID":"3df3e72f-e3b0-4213-aa87-576cab406cf8","Type":"ContainerDied","Data":"0df178f88642a99657850e8ad9e463152c07f593aebec8e8aeab444245bde49d"} Jul 20 13:15:08.338768 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:08.338720 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd" Jul 20 13:15:08.465081 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:08.464970 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df3e72f-e3b0-4213-aa87-576cab406cf8-maas-api-service-ca\") pod \"3df3e72f-e3b0-4213-aa87-576cab406cf8\" (UID: \"3df3e72f-e3b0-4213-aa87-576cab406cf8\") " Jul 20 13:15:08.465354 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:08.465329 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3df3e72f-e3b0-4213-aa87-576cab406cf8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3df3e72f-e3b0-4213-aa87-576cab406cf8" (UID: "3df3e72f-e3b0-4213-aa87-576cab406cf8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:15:08.566602 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:08.566565 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df3e72f-e3b0-4213-aa87-576cab406cf8-maas-api-service-ca\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:15:09.204116 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:09.204082 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd" Jul 20 13:15:09.204116 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:09.204088 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd" event={"ID":"3df3e72f-e3b0-4213-aa87-576cab406cf8","Type":"ContainerDied","Data":"0a155b8732b14347bc095187e46446e10940aabf3cdaf6c9f71e822e7a790a16"} Jul 20 13:15:09.204338 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:09.204139 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a155b8732b14347bc095187e46446e10940aabf3cdaf6c9f71e822e7a790a16" Jul 20 13:15:15.974115 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:15.974078 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf"] Jul 20 13:15:15.978403 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:15.978376 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-f4490cdf-29742555-jzdvf"] Jul 20 13:15:15.982547 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:15.982517 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:16.521831 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:16.521775 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89" path="/var/lib/kubelet/pods/b7d98d12-bb4f-4a26-b72a-8a8ff1cbce89/volumes" Jul 20 13:15:18.889742 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:18.889692 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:15:20.585228 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:20.585179 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd"] Jul 20 13:15:20.587954 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:20.587913 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4490cdf-sbxzd"] Jul 20 13:15:20.590978 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:20.590949 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:21.000063 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:20.999971 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:22.521814 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:22.521779 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3df3e72f-e3b0-4213-aa87-576cab406cf8" path="/var/lib/kubelet/pods/3df3e72f-e3b0-4213-aa87-576cab406cf8/volumes" Jul 20 13:15:26.907671 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:26.907634 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:27.010989 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:27.010952 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:27.159495 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:27.159402 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:27.376870 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:27.376835 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:27.510975 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:27.510887 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:27.600864 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:27.600831 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:27.703587 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:27.703552 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:33.537104 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:33.537068 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:33.609065 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:33.609025 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:33.776831 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:33.776798 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:33.888552 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:33.888516 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:33.979102 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:33.979073 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:39.697169 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:39.697130 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:39.766051 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:39.766016 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:46.474778 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:46.474677 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:46.795564 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:46.795475 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:47.016784 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:47.016751 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:49.928691 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:49.928656 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:50.044894 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:50.044860 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:50.160914 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:50.160871 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:50.330954 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:50.330869 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:50.759505 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:50.759471 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:50.859385 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:50.859346 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:54.445989 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:54.445952 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:15:55.196636 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:15:55.196595 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:16:08.470719 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:08.470682 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:16:13.295271 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:13.295235 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:16:16.609807 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:16.609760 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:16:34.707104 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:34.707066 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:16:37.587905 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:37.587867 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:16:38.021292 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.021204 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t"] Jul 20 13:16:38.021579 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.021566 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3df3e72f-e3b0-4213-aa87-576cab406cf8" containerName="revoke-keys" Jul 20 13:16:38.021632 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.021581 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3df3e72f-e3b0-4213-aa87-576cab406cf8" containerName="revoke-keys" Jul 20 13:16:38.021669 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.021655 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3df3e72f-e3b0-4213-aa87-576cab406cf8" containerName="revoke-keys" Jul 20 13:16:38.024836 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.024815 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t" Jul 20 13:16:38.026964 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.026935 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4s4l\"" Jul 20 13:16:38.027224 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.027209 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 20 13:16:38.038055 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.038022 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t"] Jul 20 13:16:38.072406 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.072323 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b3dfd35-a8b0-4bfb-8192-8f807a143a9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0288096f-j4l5t\" (UID: \"2b3dfd35-a8b0-4bfb-8192-8f807a143a9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t" Jul 20 13:16:38.173091 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.173045 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b3dfd35-a8b0-4bfb-8192-8f807a143a9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0288096f-j4l5t\" (UID: \"2b3dfd35-a8b0-4bfb-8192-8f807a143a9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t" Jul 20 13:16:38.173776 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.173755 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b3dfd35-a8b0-4bfb-8192-8f807a143a9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0288096f-j4l5t\" (UID: \"2b3dfd35-a8b0-4bfb-8192-8f807a143a9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t" Jul 20 13:16:38.334759 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.334721 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t" Jul 20 13:16:38.476709 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.476681 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t"] Jul 20 13:16:38.479378 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:16:38.479337 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3dfd35_a8b0_4bfb_8192_8f807a143a9f.slice/crio-ad94357c2b586f172e12e8b26faa4458a7452c1272c2817a7fb1b7e9b7fa2a1b WatchSource:0}: Error finding container ad94357c2b586f172e12e8b26faa4458a7452c1272c2817a7fb1b7e9b7fa2a1b: Status 404 returned error can't find the container with id ad94357c2b586f172e12e8b26faa4458a7452c1272c2817a7fb1b7e9b7fa2a1b Jul 20 13:16:38.650185 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.650099 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t" event={"ID":"2b3dfd35-a8b0-4bfb-8192-8f807a143a9f","Type":"ContainerStarted","Data":"ad94357c2b586f172e12e8b26faa4458a7452c1272c2817a7fb1b7e9b7fa2a1b"} Jul 20 13:16:38.871543 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:38.871509 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:16:39.656554 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:39.656460 2570 generic.go:356] "Generic (PLEG): container finished" podID="2b3dfd35-a8b0-4bfb-8192-8f807a143a9f" containerID="38ebd0007a03612fb80546285d850598214496b8552eb0f2f82fa4a2676d3ace" exitCode=0 Jul 20 13:16:39.656920 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:39.656551 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t" event={"ID":"2b3dfd35-a8b0-4bfb-8192-8f807a143a9f","Type":"ContainerDied","Data":"38ebd0007a03612fb80546285d850598214496b8552eb0f2f82fa4a2676d3ace"} Jul 20 13:16:40.801891 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:40.801861 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t" Jul 20 13:16:40.898867 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:40.898831 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b3dfd35-a8b0-4bfb-8192-8f807a143a9f-maas-api-service-ca\") pod \"2b3dfd35-a8b0-4bfb-8192-8f807a143a9f\" (UID: \"2b3dfd35-a8b0-4bfb-8192-8f807a143a9f\") " Jul 20 13:16:40.899267 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:40.899244 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b3dfd35-a8b0-4bfb-8192-8f807a143a9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2b3dfd35-a8b0-4bfb-8192-8f807a143a9f" (UID: "2b3dfd35-a8b0-4bfb-8192-8f807a143a9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:16:40.999717 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:40.999613 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b3dfd35-a8b0-4bfb-8192-8f807a143a9f-maas-api-service-ca\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:16:41.667748 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:41.667717 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t" Jul 20 13:16:41.667748 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:41.667738 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t" event={"ID":"2b3dfd35-a8b0-4bfb-8192-8f807a143a9f","Type":"ContainerDied","Data":"ad94357c2b586f172e12e8b26faa4458a7452c1272c2817a7fb1b7e9b7fa2a1b"} Jul 20 13:16:41.667985 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:41.667780 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad94357c2b586f172e12e8b26faa4458a7452c1272c2817a7fb1b7e9b7fa2a1b" Jul 20 13:16:52.309072 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:52.309037 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:16:53.630266 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:53.630236 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:16:58.132566 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:58.132528 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:16:58.135086 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:58.133591 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t"] Jul 20 13:16:58.136576 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:58.136550 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0288096f-j4l5t"] Jul 20 13:16:58.521726 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:16:58.521641 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b3dfd35-a8b0-4bfb-8192-8f807a143a9f" path="/var/lib/kubelet/pods/2b3dfd35-a8b0-4bfb-8192-8f807a143a9f/volumes" Jul 20 13:17:00.328591 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:00.328548 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:01.101815 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:01.101779 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:17:05.408645 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:05.408604 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:09.625658 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:09.625616 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:09.720070 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:09.720039 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:41.987348 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:41.987254 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:42.762452 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:42.762391 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:48.564989 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:48.564950 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:48.661776 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:48.661736 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:49.050632 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:49.050589 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:49.163276 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:49.163237 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:50.306760 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:50.306707 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:50.511391 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:50.511346 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:51.598104 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:51.598060 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:51.959179 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:51.959077 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:17:53.432614 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:53.432578 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:17:53.524667 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:17:53.524633 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:01.602766 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:01.602714 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:01.849293 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:01.849258 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:02.072148 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:02.072061 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:05.863270 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:05.863224 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:18:10.091069 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:10.091034 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:10.199943 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:10.199908 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:11.128302 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:11.128268 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:11.372873 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:11.372841 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:18.691680 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:18.691642 2570 scope.go:117] "RemoveContainer" containerID="d80ab603823aa804b02b31245c300cc80b5b0bd3170a3946ec1de9c44806d336" Jul 20 13:18:21.875805 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:21.875763 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:22.220134 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:22.220041 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:39.474838 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:39.474799 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:43.252293 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:43.252201 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:50.429330 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:50.429281 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:50.529246 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:50.529215 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:50.793965 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:50.793869 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:51.267031 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:51.266982 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:51.392624 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:51.392581 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:53.920944 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:53.920895 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:54.038004 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:54.037946 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:54.190265 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:54.190173 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:54.633944 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:54.633910 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:54.818497 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:54.818452 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:54.923852 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:54.923763 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:58.377888 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:58.377850 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:58.711738 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:58.711645 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:59.001126 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.001038 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln"] Jul 20 13:18:59.001420 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.001396 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b3dfd35-a8b0-4bfb-8192-8f807a143a9f" containerName="revoke-keys" Jul 20 13:18:59.001420 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.001422 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b3dfd35-a8b0-4bfb-8192-8f807a143a9f" containerName="revoke-keys" Jul 20 13:18:59.001552 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.001493 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b3dfd35-a8b0-4bfb-8192-8f807a143a9f" containerName="revoke-keys" Jul 20 13:18:59.004631 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.004611 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.007060 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.007040 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1feb1a-openshift-default-dockercfg-67k26\"" Jul 20 13:18:59.019473 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.019437 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln"] Jul 20 13:18:59.030939 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.030896 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:59.052215 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.052178 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-credential-socket\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.052419 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.052243 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-token\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.052419 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.052313 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-certs\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.052419 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.052340 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmttn\" (UniqueName: \"kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-kube-api-access-lmttn\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.052419 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.052358 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-data\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.052419 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.052395 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-podinfo\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.052419 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.052415 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istiod-ca-cert\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.052688 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.052545 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-envoy\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.052688 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.052598 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-socket\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.128561 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.128520 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:18:59.153985 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.153944 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-certs\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.153985 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.153990 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmttn\" (UniqueName: \"kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-kube-api-access-lmttn\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.154264 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.154018 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-data\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.154264 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.154071 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-podinfo\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.154264 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.154123 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istiod-ca-cert\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.154264 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.154189 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-envoy\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.154264 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.154239 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-socket\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.154557 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.154272 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-credential-socket\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.154557 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.154323 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-token\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.154557 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.154498 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-data\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.154839 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.154815 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-socket\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.154924 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.154818 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-credential-socket\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.155064 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.155006 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-certs\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.155247 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.155228 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istiod-ca-cert\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.156929 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.156883 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-podinfo\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.156929 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.156881 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-envoy\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.166625 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.166592 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-token\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.166787 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.166724 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmttn\" (UniqueName: \"kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-kube-api-access-lmttn\") pod \"e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.315218 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.315125 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:18:59.421925 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.421881 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:18:59.472431 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.472403 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln"] Jul 20 13:18:59.474042 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:18:59.474017 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab8ff3cc_b2cc_4b10_be93_7e9fdb4bad8c.slice/crio-069bc3d6127d2a427b24cdd977aad7f06f0941884e55a79e636c3c1cebde11b8 WatchSource:0}: Error finding container 069bc3d6127d2a427b24cdd977aad7f06f0941884e55a79e636c3c1cebde11b8: Status 404 returned error can't find the container with id 069bc3d6127d2a427b24cdd977aad7f06f0941884e55a79e636c3c1cebde11b8 Jul 20 13:18:59.476171 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.476137 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:18:59.476246 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.476215 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:18:59.476285 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:18:59.476246 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:19:00.348003 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:00.347951 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" event={"ID":"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c","Type":"ContainerStarted","Data":"362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06"} Jul 20 13:19:00.348199 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:00.348013 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" event={"ID":"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c","Type":"ContainerStarted","Data":"069bc3d6127d2a427b24cdd977aad7f06f0941884e55a79e636c3c1cebde11b8"} Jul 20 13:19:00.369737 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:00.369679 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" podStartSLOduration=2.36966434 podStartE2EDuration="2.36966434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:18:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:19:00.367803384 +0000 UTC m=+2082.340009240" watchObservedRunningTime="2026-07-20 13:19:00.36966434 +0000 UTC m=+2082.341870172" Jul 20 13:19:01.315764 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:01.315727 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:19:01.321488 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:01.321459 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:19:01.353675 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:01.353645 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:19:01.354605 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:01.354583 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:19:02.397895 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:02.397859 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:19:02.507161 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:02.507126 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:19:02.721135 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:02.721040 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:19:02.814341 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:02.814312 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:19:03.458971 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:03.458936 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:19:03.645795 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:03.645751 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:19:06.864295 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:06.864258 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:19:08.513686 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:08.513651 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:19:15.740160 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:15.740122 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:19:15.838938 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:15.838895 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:19:16.827057 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:16.827001 2570 patch_prober.go:28] interesting pod/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": read tcp 10.133.0.2:57824->10.133.0.43:15021: read: connection reset by peer" start-of-body= Jul 20 13:19:16.827477 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:16.827086 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": read tcp 10.133.0.2:57824->10.133.0.43:15021: read: connection reset by peer" Jul 20 13:19:17.432970 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:17.432935 2570 generic.go:356] "Generic (PLEG): container finished" podID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerID="362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06" exitCode=0 Jul 20 13:19:17.433148 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:17.432977 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" event={"ID":"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c","Type":"ContainerDied","Data":"362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06"} Jul 20 13:19:17.433355 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:17.433343 2570 scope.go:117] "RemoveContainer" containerID="362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06" Jul 20 13:19:17.434489 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:17.434451 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:19:17.434605 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:17.434522 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:19:17.434605 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:17.434559 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:19:18.445760 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:18.444762 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" event={"ID":"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c","Type":"ContainerStarted","Data":"bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252"} Jul 20 13:19:18.626248 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:18.626216 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:19:18.629674 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:18.629648 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:19:18.711809 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:18.711710 2570 scope.go:117] "RemoveContainer" containerID="f689d9c88b8eb78185f7f585179db428abd6ac384e480c179cfe4e3a15a38c86" Jul 20 13:19:19.315346 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:19.315309 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:19:19.315637 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:19.315615 2570 patch_prober.go:28] interesting pod/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 20 13:19:19.315699 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:19.315678 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 20 13:19:19.402353 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:19.402308 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:19:20.316459 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:20.316416 2570 patch_prober.go:28] interesting pod/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 20 13:19:20.316878 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:20.316488 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 20 13:19:21.315824 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:21.315787 2570 patch_prober.go:28] interesting pod/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 20 13:19:21.316035 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:21.315846 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 20 13:19:22.315900 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:22.315858 2570 patch_prober.go:28] interesting pod/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 20 13:19:22.316309 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:22.315918 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 20 13:19:23.320241 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:23.320203 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:19:23.320718 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:23.320623 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:19:23.321607 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:23.321589 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:19:26.256425 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:26.256387 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:19:26.375842 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:26.375809 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:19:34.798782 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:19:34.798734 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:20:03.345091 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:20:03.345037 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:20:08.831038 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:20:08.830993 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:20:42.038566 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:20:42.038479 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:20:43.086954 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:20:43.086920 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:20:43.812658 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:20:43.812617 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:20:43.913207 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:20:43.913170 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:20:47.108541 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:20:47.108500 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:20:53.825564 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:20:53.825523 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:20:58.263593 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:20:58.263552 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:20:58.703711 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:20:58.703672 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:02.900525 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:02.900487 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:21:03.329124 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:03.329030 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:03.394986 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:03.394950 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:04.784181 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:04.784147 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:04.889901 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:04.889857 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:05.119567 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:05.119533 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:05.314407 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:05.314351 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:06.226042 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:06.226002 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:06.363697 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:06.363664 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:16.154317 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:16.154259 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:18.739719 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:18.739686 2570 scope.go:117] "RemoveContainer" containerID="f343dd11e76eb2afb47a791c349f53e04a8dd16a5ab8931c4e099e7e2fb20a5c" Jul 20 13:21:18.748868 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:18.748845 2570 scope.go:117] "RemoveContainer" containerID="0df178f88642a99657850e8ad9e463152c07f593aebec8e8aeab444245bde49d" Jul 20 13:21:19.110325 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:19.110279 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:21:20.568660 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:20.568624 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:21.003547 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:21.003510 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:27.344525 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:27.344475 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:27.444641 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:27.444611 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:27.592689 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:27.592656 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln"] Jul 20 13:21:27.592901 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:27.592878 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" containerID="cri-o://bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252" gracePeriod=30 Jul 20 13:21:27.619694 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:27.619591 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:27.874281 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:27.874193 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:28.259049 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:28.258951 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:28.621007 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:28.620967 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:29.326968 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:29.326932 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:31.626759 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:31.626715 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:31.752634 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:31.752593 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:31.877545 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:31.877450 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:31.993222 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:31.993160 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:32.552799 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.552763 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:32.652438 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.652402 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:32.861192 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.861158 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:21:32.998247 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998209 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-credential-socket\") pod \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " Jul 20 13:21:32.998247 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998260 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-token\") pod \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " Jul 20 13:21:32.998586 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998296 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmttn\" (UniqueName: \"kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-kube-api-access-lmttn\") pod \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " Jul 20 13:21:32.998586 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998320 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-socket\") pod \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " Jul 20 13:21:32.998586 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998348 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-podinfo\") pod \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " Jul 20 13:21:32.998586 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998417 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istiod-ca-cert\") pod \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " Jul 20 13:21:32.998586 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998447 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-envoy\") pod \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " Jul 20 13:21:32.998586 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998474 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-data\") pod \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " Jul 20 13:21:32.998586 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998506 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-certs\") pod \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\" (UID: \"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c\") " Jul 20 13:21:32.998972 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998726 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" (UID: "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:21:32.998972 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998871 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-socket\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:21:32.999081 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998978 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" (UID: "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:21:32.999081 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.998988 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" (UID: "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:21:32.999081 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.999063 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" (UID: "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:21:32.999535 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:32.999514 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" (UID: "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:21:33.000721 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.000696 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" (UID: "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 13:21:33.000833 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.000739 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" (UID: "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:21:33.001023 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.001007 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-kube-api-access-lmttn" (OuterVolumeSpecName: "kube-api-access-lmttn") pod "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" (UID: "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c"). InnerVolumeSpecName "kube-api-access-lmttn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:21:33.001023 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.001012 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" (UID: "ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:21:33.100092 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.100005 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-credential-socket\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:21:33.100092 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.100036 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-token\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:21:33.100092 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.100046 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmttn\" (UniqueName: \"kubernetes.io/projected/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-kube-api-access-lmttn\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:21:33.100092 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.100055 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-podinfo\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:21:33.100092 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.100065 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istiod-ca-cert\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:21:33.100092 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.100075 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-envoy\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:21:33.100092 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.100082 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-istio-data\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:21:33.100092 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.100089 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c-workload-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:21:33.133818 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.133786 2570 generic.go:356] "Generic (PLEG): container finished" podID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerID="bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252" exitCode=0 Jul 20 13:21:33.133994 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.133837 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" Jul 20 13:21:33.133994 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.133838 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" event={"ID":"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c","Type":"ContainerDied","Data":"bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252"} Jul 20 13:21:33.133994 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.133969 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln" event={"ID":"ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c","Type":"ContainerDied","Data":"069bc3d6127d2a427b24cdd977aad7f06f0941884e55a79e636c3c1cebde11b8"} Jul 20 13:21:33.134115 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.133998 2570 scope.go:117] "RemoveContainer" containerID="bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252" Jul 20 13:21:33.144934 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.144907 2570 scope.go:117] "RemoveContainer" containerID="362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06" Jul 20 13:21:33.153753 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.153730 2570 scope.go:117] "RemoveContainer" containerID="bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252" Jul 20 13:21:33.154058 ip-10-0-141-229 kubenswrapper[2570]: E0720 13:21:33.154037 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252\": container with ID starting with bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252 not found: ID does not exist" containerID="bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252" Jul 20 13:21:33.154104 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.154070 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252"} err="failed to get container status \"bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252\": rpc error: code = NotFound desc = could not find container \"bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252\": container with ID starting with bf7c32a6d4df28f7609aa1566d6004b46d4fd668ccd7f691b0c898053148b252 not found: ID does not exist" Jul 20 13:21:33.154104 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.154092 2570 scope.go:117] "RemoveContainer" containerID="362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06" Jul 20 13:21:33.154409 ip-10-0-141-229 kubenswrapper[2570]: E0720 13:21:33.154355 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06\": container with ID starting with 362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06 not found: ID does not exist" containerID="362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06" Jul 20 13:21:33.154515 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.154421 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06"} err="failed to get container status \"362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06\": rpc error: code = NotFound desc = could not find container \"362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06\": container with ID starting with 362872fda8efc610802c9d79a0e1cb63b3b0c2c9211453ba4468337363b6da06 not found: ID does not exist" Jul 20 13:21:33.163486 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.163450 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln"] Jul 20 13:21:33.167253 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:33.167220 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1feb1a-openshift-default-7dc5588675-g87ln"] Jul 20 13:21:34.521916 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:34.521883 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" path="/var/lib/kubelet/pods/ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c/volumes" Jul 20 13:21:36.094383 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:36.094329 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:36.620056 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:36.620026 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:36.912795 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:36.912699 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb"] Jul 20 13:21:36.913213 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:36.913199 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" Jul 20 13:21:36.913257 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:36.913218 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" Jul 20 13:21:36.913316 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:36.913305 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" Jul 20 13:21:36.913356 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:36.913323 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" Jul 20 13:21:36.917253 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:36.917226 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:36.920264 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:36.920237 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1797d3-openshift-default-dockercfg-sz4rv\"" Jul 20 13:21:36.931446 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:36.931415 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb"] Jul 20 13:21:36.963636 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:36.963595 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:37.035207 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.035169 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-podinfo\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.035207 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.035219 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-token\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.035469 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.035255 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-data\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.035469 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.035316 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw8n7\" (UniqueName: \"kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-kube-api-access-mw8n7\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.035469 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.035415 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-certs\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.035469 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.035443 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-credential-socket\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.035469 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.035470 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istiod-ca-cert\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.035755 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.035497 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-envoy\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.035755 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.035525 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-socket\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.136323 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.136280 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-podinfo\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.136860 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.136336 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-token\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.136860 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.136410 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-data\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.136860 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.136436 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw8n7\" (UniqueName: \"kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-kube-api-access-mw8n7\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.136860 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.136505 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-certs\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.136860 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.136542 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-credential-socket\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.136860 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.136568 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istiod-ca-cert\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.136860 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.136605 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-envoy\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.136860 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.136645 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-socket\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.136860 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.136855 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-data\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.137345 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.137063 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-certs\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.137534 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.137474 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-socket\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.137692 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.137671 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-credential-socket\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.137922 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.137894 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istiod-ca-cert\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.139301 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.139277 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-envoy\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.139459 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.139443 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-podinfo\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.151773 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.151739 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw8n7\" (UniqueName: \"kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-kube-api-access-mw8n7\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.153531 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.153502 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-token\") pod \"e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.195242 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.195127 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:37.228172 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.228136 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:37.386253 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.386094 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb"] Jul 20 13:21:37.389168 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:21:37.389138 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41d230a7_79f3_4b1f_8c8b_76f0a69b34ad.slice/crio-6c89f98a09b31dfdf3ba89679587961376c7cee33eeab9015aa65f0f90ea0db5 WatchSource:0}: Error finding container 6c89f98a09b31dfdf3ba89679587961376c7cee33eeab9015aa65f0f90ea0db5: Status 404 returned error can't find the container with id 6c89f98a09b31dfdf3ba89679587961376c7cee33eeab9015aa65f0f90ea0db5 Jul 20 13:21:37.391299 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.391265 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:21:37.391382 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.391342 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:21:37.391430 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:37.391387 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:21:38.163440 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:38.163401 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" event={"ID":"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad","Type":"ContainerStarted","Data":"4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3"} Jul 20 13:21:38.163872 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:38.163460 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" event={"ID":"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad","Type":"ContainerStarted","Data":"6c89f98a09b31dfdf3ba89679587961376c7cee33eeab9015aa65f0f90ea0db5"} Jul 20 13:21:38.195564 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:38.195504 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" podStartSLOduration=2.195486112 podStartE2EDuration="2.195486112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:21:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:21:38.193948861 +0000 UTC m=+2240.166154700" watchObservedRunningTime="2026-07-20 13:21:38.195486112 +0000 UTC m=+2240.167691943" Jul 20 13:21:38.229052 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:38.229015 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:38.234484 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:38.234454 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:38.826886 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:38.826841 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:21:39.168355 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:39.168315 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:39.169264 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:39.169241 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:40.191568 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:40.191530 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:42.051353 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:42.051322 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:44.700681 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:44.700652 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:53.464247 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:53.464199 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:53.564274 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:53.564222 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:21:55.209619 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:55.209567 2570 patch_prober.go:28] interesting pod/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 13:21:55.210116 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:55.209663 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 13:21:56.252384 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:56.252340 2570 generic.go:356] "Generic (PLEG): container finished" podID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerID="4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3" exitCode=0 Jul 20 13:21:56.252808 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:56.252413 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" event={"ID":"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad","Type":"ContainerDied","Data":"4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3"} Jul 20 13:21:56.252851 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:56.252827 2570 scope.go:117] "RemoveContainer" containerID="4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3" Jul 20 13:21:56.253869 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:56.253837 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:21:56.253992 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:56.253910 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:21:56.253992 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:56.253941 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 13:21:56.600283 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:56.600234 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:21:57.263362 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:57.263319 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" event={"ID":"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad","Type":"ContainerStarted","Data":"3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5"} Jul 20 13:21:58.228721 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:58.228674 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:21:58.229033 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:58.229008 2570 patch_prober.go:28] interesting pod/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 20 13:21:58.229082 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:58.229059 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 20 13:21:59.229025 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:59.228985 2570 patch_prober.go:28] interesting pod/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 20 13:21:59.229458 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:21:59.229045 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 20 13:22:00.229218 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:00.229176 2570 patch_prober.go:28] interesting pod/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 20 13:22:00.229638 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:00.229234 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 20 13:22:01.233559 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:01.233527 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:22:01.234031 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:01.233809 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:22:01.234688 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:01.234665 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:22:04.156678 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:04.156644 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:04.260055 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:04.260001 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:09.438383 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:09.438328 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:12.882434 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:12.882389 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:13.259589 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:13.259496 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:14.511600 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:14.511567 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:22:19.253261 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:19.253222 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:19.354182 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:19.354146 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:19.532400 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:19.532292 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:19.736551 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:19.736508 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:35.615391 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:35.615330 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:39.956013 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:39.955973 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:40.378050 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:40.378021 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:40.917348 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:40.917312 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:22:46.698113 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:46.698071 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:46.800091 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:46.800047 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:46.949523 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:46.949428 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb"] Jul 20 13:22:46.949710 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:46.949685 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerName="istio-proxy" containerID="cri-o://3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5" gracePeriod=30 Jul 20 13:22:46.989089 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:46.989049 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:47.236143 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:47.236044 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:47.561714 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:47.561616 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:47.900964 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:47.900931 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:48.040771 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:48.040736 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:51.020024 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:51.019985 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:51.116974 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:51.116943 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:51.217439 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:51.217401 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:51.323252 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:51.323165 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:52.204732 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.204696 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:52.219334 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.219265 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:22:52.312620 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.312588 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:52.342254 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342215 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-envoy\") pod \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " Jul 20 13:22:52.342477 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342320 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-podinfo\") pod \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " Jul 20 13:22:52.342477 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342347 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-credential-socket\") pod \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " Jul 20 13:22:52.342477 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342391 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-socket\") pod \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " Jul 20 13:22:52.342477 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342420 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-data\") pod \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " Jul 20 13:22:52.342477 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342455 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istiod-ca-cert\") pod \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " Jul 20 13:22:52.342749 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342494 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw8n7\" (UniqueName: \"kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-kube-api-access-mw8n7\") pod \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " Jul 20 13:22:52.342749 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342515 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-certs\") pod \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " Jul 20 13:22:52.342749 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342543 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-token\") pod \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\" (UID: \"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad\") " Jul 20 13:22:52.342926 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342787 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" (UID: "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:22:52.342926 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342871 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-socket\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:22:52.343036 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.342926 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" (UID: "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:22:52.343089 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.343066 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" (UID: "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:22:52.343316 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.343282 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" (UID: "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:22:52.343481 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.343419 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-data" (OuterVolumeSpecName: "istio-data") pod "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" (UID: "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:22:52.345157 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.345131 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" (UID: "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 13:22:52.345267 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.345185 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" (UID: "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:22:52.345392 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.345347 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-token" (OuterVolumeSpecName: "istio-token") pod "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" (UID: "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:22:52.345445 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.345416 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-kube-api-access-mw8n7" (OuterVolumeSpecName: "kube-api-access-mw8n7") pod "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" (UID: "41d230a7-79f3-4b1f-8c8b-76f0a69b34ad"). InnerVolumeSpecName "kube-api-access-mw8n7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:22:52.443522 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.443480 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-podinfo\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:22:52.443522 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.443521 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-credential-socket\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:22:52.443774 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.443536 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-data\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:22:52.443774 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.443549 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istiod-ca-cert\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:22:52.443774 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.443562 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw8n7\" (UniqueName: \"kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-kube-api-access-mw8n7\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:22:52.443774 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.443579 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-workload-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:22:52.443774 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.443595 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-token\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:22:52.443774 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.443607 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad-istio-envoy\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:22:52.539967 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.539908 2570 generic.go:356] "Generic (PLEG): container finished" podID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerID="3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5" exitCode=0 Jul 20 13:22:52.540237 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.539977 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" event={"ID":"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad","Type":"ContainerDied","Data":"3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5"} Jul 20 13:22:52.540237 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.540085 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" event={"ID":"41d230a7-79f3-4b1f-8c8b-76f0a69b34ad","Type":"ContainerDied","Data":"6c89f98a09b31dfdf3ba89679587961376c7cee33eeab9015aa65f0f90ea0db5"} Jul 20 13:22:52.540237 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.540132 2570 scope.go:117] "RemoveContainer" containerID="3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5" Jul 20 13:22:52.541880 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.541856 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb" Jul 20 13:22:52.551559 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.551535 2570 scope.go:117] "RemoveContainer" containerID="4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3" Jul 20 13:22:52.560067 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.560039 2570 scope.go:117] "RemoveContainer" containerID="3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5" Jul 20 13:22:52.560630 ip-10-0-141-229 kubenswrapper[2570]: E0720 13:22:52.560441 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5\": container with ID starting with 3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5 not found: ID does not exist" containerID="3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5" Jul 20 13:22:52.560630 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.560477 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5"} err="failed to get container status \"3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5\": rpc error: code = NotFound desc = could not find container \"3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5\": container with ID starting with 3c30c7b4a86e03269ffd73bf2844736676de8fb28dbed00fc27b4cbb56a351d5 not found: ID does not exist" Jul 20 13:22:52.560630 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.560498 2570 scope.go:117] "RemoveContainer" containerID="4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3" Jul 20 13:22:52.560836 ip-10-0-141-229 kubenswrapper[2570]: E0720 13:22:52.560734 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3\": container with ID starting with 4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3 not found: ID does not exist" containerID="4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3" Jul 20 13:22:52.560836 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.560758 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3"} err="failed to get container status \"4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3\": rpc error: code = NotFound desc = could not find container \"4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3\": container with ID starting with 4e964bb59f28269bcd32a10503bc329b4428b1a90df32c1fcc4e61cfb8683bf3 not found: ID does not exist" Jul 20 13:22:52.649434 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.649398 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb"] Jul 20 13:22:52.656680 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:52.656647 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1797d3-openshift-default-c564c98b7-xnzlb"] Jul 20 13:22:54.521539 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:54.521497 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" path="/var/lib/kubelet/pods/41d230a7-79f3-4b1f-8c8b-76f0a69b34ad/volumes" Jul 20 13:22:55.528866 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:55.528818 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:55.852273 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:55.852232 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:56.211289 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:56.211166 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:56.376407 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:56.376339 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:56.705866 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:56.705825 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:22:59.074284 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:59.074244 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:59.169988 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:59.169953 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:59.271072 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:59.271036 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:59.641027 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:59.640984 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:59.817693 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:59.817657 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:22:59.918490 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:22:59.918403 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:00.609508 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:00.609471 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:03.551346 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:03.551306 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:09.198784 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:09.198739 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:23:11.163680 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:11.163595 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:11.316179 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:11.316142 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:11.611539 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:11.611500 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:12.870072 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:12.870039 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:12.970674 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:12.970635 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:18.782186 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:18.782149 2570 scope.go:117] "RemoveContainer" containerID="38ebd0007a03612fb80546285d850598214496b8552eb0f2f82fa4a2676d3ace" Jul 20 13:23:19.925554 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:19.925519 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:20.704627 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:20.704584 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:20.808082 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:20.808046 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:23.314859 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:23.314815 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:23:44.229307 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:44.229272 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:23:47.508692 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:23:47.508652 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:24:02.307349 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:24:02.307308 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:24:18.678007 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:24:18.677969 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:24:18.682129 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:24:18.682102 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:24:20.610543 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:24:20.610510 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:24:39.695944 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:24:39.695900 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:24:59.932032 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:24:59.931993 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:25:19.325836 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:25:19.325793 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:25:39.758131 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:25:39.758096 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:25:58.038393 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:25:58.038179 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:26:02.855800 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:02.855760 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:26:03.762445 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:03.762391 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:26:07.180526 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:07.180491 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:26:14.442419 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:14.442379 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:26:14.515905 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:14.515865 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:26:15.405626 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:15.405587 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:26:20.518494 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:20.518455 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:26:26.395336 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:26.395298 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:26:55.088409 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.088355 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6"] Jul 20 13:26:55.088998 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.088868 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" Jul 20 13:26:55.088998 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.088890 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8ff3cc-b2cc-4b10-be93-7e9fdb4bad8c" containerName="istio-proxy" Jul 20 13:26:55.088998 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.088908 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerName="istio-proxy" Jul 20 13:26:55.088998 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.088916 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerName="istio-proxy" Jul 20 13:26:55.089206 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.089020 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerName="istio-proxy" Jul 20 13:26:55.089206 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.089037 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerName="istio-proxy" Jul 20 13:26:55.092201 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.092177 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.095013 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.094971 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b8383c\"/\"kube-root-ca.crt\"" Jul 20 13:26:55.095594 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.095571 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b8383c\"/\"default-dockercfg-c88sc\"" Jul 20 13:26:55.095714 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.095643 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b8383c\"/\"openshift-service-ca.crt\"" Jul 20 13:26:55.095714 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.095663 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b8383c\"/\"sub-test-model-b8383c-kserve-self-signed-certs\"" Jul 20 13:26:55.105255 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.105230 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6"] Jul 20 13:26:55.115649 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.115615 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:26:55.258786 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.258738 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-model-cache\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.258976 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.258844 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-dshm\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.258976 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.258874 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-home\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.258976 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.258960 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tmp-dir\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.259095 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.259001 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tls-certs\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.259095 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.259061 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccd52\" (UniqueName: \"kubernetes.io/projected/356230fa-7bf5-4915-ba1b-04d5f16b65c5-kube-api-access-ccd52\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.360076 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.359989 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-dshm\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.360076 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.360029 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-home\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.360076 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.360058 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tmp-dir\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.360347 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.360081 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tls-certs\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.360347 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.360116 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccd52\" (UniqueName: \"kubernetes.io/projected/356230fa-7bf5-4915-ba1b-04d5f16b65c5-kube-api-access-ccd52\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.360347 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.360141 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-model-cache\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.360598 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.360473 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-home\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.360598 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.360562 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tmp-dir\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.360598 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.360591 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-model-cache\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.362319 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.362299 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-dshm\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.362591 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.362576 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tls-certs\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.368742 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.368719 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccd52\" (UniqueName: \"kubernetes.io/projected/356230fa-7bf5-4915-ba1b-04d5f16b65c5-kube-api-access-ccd52\") pod \"sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.401421 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.401385 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:26:55.555135 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.545053 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6"] Jul 20 13:26:55.555135 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:26:55.547930 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod356230fa_7bf5_4915_ba1b_04d5f16b65c5.slice/crio-640db1aa5b7cd9c140e5d2b175ac66eecce2746fbf00c4e24f6b573d0b7bb006 WatchSource:0}: Error finding container 640db1aa5b7cd9c140e5d2b175ac66eecce2746fbf00c4e24f6b573d0b7bb006: Status 404 returned error can't find the container with id 640db1aa5b7cd9c140e5d2b175ac66eecce2746fbf00c4e24f6b573d0b7bb006 Jul 20 13:26:55.700794 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.700695 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" event={"ID":"356230fa-7bf5-4915-ba1b-04d5f16b65c5","Type":"ContainerStarted","Data":"6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0"} Jul 20 13:26:55.700794 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.700742 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" event={"ID":"356230fa-7bf5-4915-ba1b-04d5f16b65c5","Type":"ContainerStarted","Data":"640db1aa5b7cd9c140e5d2b175ac66eecce2746fbf00c4e24f6b573d0b7bb006"} Jul 20 13:26:55.721356 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:55.721295 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" podStartSLOduration=0.721280217 podStartE2EDuration="721.280217ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:26:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:26:55.719588518 +0000 UTC m=+2557.691794350" watchObservedRunningTime="2026-07-20 13:26:55.721280217 +0000 UTC m=+2557.693486048" Jul 20 13:26:59.214088 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:26:59.214049 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:27:05.402474 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:05.402348 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:27:05.402876 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:05.402500 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:27:05.415616 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:05.415588 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:27:05.758729 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:05.758638 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:27:11.054974 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:11.054927 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:27:26.249110 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:26.249074 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:27:28.186615 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:28.186577 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6"] Jul 20 13:27:28.249159 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:28.249108 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:27:32.866170 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:32.866128 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:27:41.040443 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:41.040344 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:27:43.207299 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.207256 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" podUID="356230fa-7bf5-4915-ba1b-04d5f16b65c5" containerName="main" containerID="cri-o://6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0" gracePeriod=45 Jul 20 13:27:43.463228 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.463150 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:27:43.621271 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.621231 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccd52\" (UniqueName: \"kubernetes.io/projected/356230fa-7bf5-4915-ba1b-04d5f16b65c5-kube-api-access-ccd52\") pod \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " Jul 20 13:27:43.621500 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.621292 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tmp-dir\") pod \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " Jul 20 13:27:43.621500 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.621331 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tls-certs\") pod \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " Jul 20 13:27:43.621500 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.621386 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-dshm\") pod \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " Jul 20 13:27:43.621500 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.621439 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-home\") pod \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " Jul 20 13:27:43.621500 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.621466 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-model-cache\") pod \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\" (UID: \"356230fa-7bf5-4915-ba1b-04d5f16b65c5\") " Jul 20 13:27:43.621787 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.621567 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "356230fa-7bf5-4915-ba1b-04d5f16b65c5" (UID: "356230fa-7bf5-4915-ba1b-04d5f16b65c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:27:43.621787 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.621725 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-home" (OuterVolumeSpecName: "home") pod "356230fa-7bf5-4915-ba1b-04d5f16b65c5" (UID: "356230fa-7bf5-4915-ba1b-04d5f16b65c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:27:43.621787 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.621750 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:27:43.621787 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.621754 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-model-cache" (OuterVolumeSpecName: "model-cache") pod "356230fa-7bf5-4915-ba1b-04d5f16b65c5" (UID: "356230fa-7bf5-4915-ba1b-04d5f16b65c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:27:43.623667 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.623637 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-dshm" (OuterVolumeSpecName: "dshm") pod "356230fa-7bf5-4915-ba1b-04d5f16b65c5" (UID: "356230fa-7bf5-4915-ba1b-04d5f16b65c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:27:43.623790 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.623715 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/356230fa-7bf5-4915-ba1b-04d5f16b65c5-kube-api-access-ccd52" (OuterVolumeSpecName: "kube-api-access-ccd52") pod "356230fa-7bf5-4915-ba1b-04d5f16b65c5" (UID: "356230fa-7bf5-4915-ba1b-04d5f16b65c5"). InnerVolumeSpecName "kube-api-access-ccd52". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:27:43.623790 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.623721 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "356230fa-7bf5-4915-ba1b-04d5f16b65c5" (UID: "356230fa-7bf5-4915-ba1b-04d5f16b65c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:27:43.722859 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.722755 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:27:43.722859 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.722796 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:27:43.722859 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.722807 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccd52\" (UniqueName: \"kubernetes.io/projected/356230fa-7bf5-4915-ba1b-04d5f16b65c5-kube-api-access-ccd52\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:27:43.722859 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.722816 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/356230fa-7bf5-4915-ba1b-04d5f16b65c5-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:27:43.722859 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.722824 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/356230fa-7bf5-4915-ba1b-04d5f16b65c5-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:27:43.928076 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.928027 2570 generic.go:356] "Generic (PLEG): container finished" podID="356230fa-7bf5-4915-ba1b-04d5f16b65c5" containerID="6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0" exitCode=0 Jul 20 13:27:43.928076 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.928079 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" Jul 20 13:27:43.928315 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.928103 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" event={"ID":"356230fa-7bf5-4915-ba1b-04d5f16b65c5","Type":"ContainerDied","Data":"6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0"} Jul 20 13:27:43.928315 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.928142 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6" event={"ID":"356230fa-7bf5-4915-ba1b-04d5f16b65c5","Type":"ContainerDied","Data":"640db1aa5b7cd9c140e5d2b175ac66eecce2746fbf00c4e24f6b573d0b7bb006"} Jul 20 13:27:43.928315 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.928164 2570 scope.go:117] "RemoveContainer" containerID="6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0" Jul 20 13:27:43.938196 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.938172 2570 scope.go:117] "RemoveContainer" containerID="6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0" Jul 20 13:27:43.938558 ip-10-0-141-229 kubenswrapper[2570]: E0720 13:27:43.938533 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0\": container with ID starting with 6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0 not found: ID does not exist" containerID="6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0" Jul 20 13:27:43.938643 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.938570 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0"} err="failed to get container status \"6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0\": rpc error: code = NotFound desc = could not find container \"6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0\": container with ID starting with 6715f2f8c941396f2614be69758ab212dedf8ab1b7208fcdb47726ddf3717eb0 not found: ID does not exist" Jul 20 13:27:43.950842 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.950807 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6"] Jul 20 13:27:43.959238 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:43.959206 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b8383c/sub-test-model-b8383c-kserve-cfb5b8cd5-2c2t6"] Jul 20 13:27:44.521710 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:44.521676 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="356230fa-7bf5-4915-ba1b-04d5f16b65c5" path="/var/lib/kubelet/pods/356230fa-7bf5-4915-ba1b-04d5f16b65c5/volumes" Jul 20 13:27:46.535058 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:27:46.535024 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:28:02.284278 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:02.284244 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:28:05.929670 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:05.929629 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:28:12.312627 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:12.312591 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:28:12.418700 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:12.418666 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:28:14.073570 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:14.073537 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:28:14.364703 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:14.364670 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:28:20.819518 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:20.819480 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:28:21.747157 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:21.747121 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:28:21.913405 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:21.913354 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:28:40.306166 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:40.306124 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:28:46.736792 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:46.736737 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:28:47.547031 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:47.546983 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:28:48.606275 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.606235 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7675-140232305502768"] Jul 20 13:28:48.606701 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.606589 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerName="istio-proxy" Jul 20 13:28:48.606701 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.606601 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="41d230a7-79f3-4b1f-8c8b-76f0a69b34ad" containerName="istio-proxy" Jul 20 13:28:48.606701 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.606613 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="356230fa-7bf5-4915-ba1b-04d5f16b65c5" containerName="main" Jul 20 13:28:48.606701 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.606618 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="356230fa-7bf5-4915-ba1b-04d5f16b65c5" containerName="main" Jul 20 13:28:48.606701 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.606682 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="356230fa-7bf5-4915-ba1b-04d5f16b65c5" containerName="main" Jul 20 13:28:48.609717 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.609695 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232305502768" Jul 20 13:28:48.611539 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.611521 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-j6xxs\"" Jul 20 13:28:48.616263 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.616232 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7675-140232305502768"] Jul 20 13:28:48.699019 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.698980 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcptg\" (UniqueName: \"kubernetes.io/projected/dfcff708-1cf2-4839-872a-a1300967ad93-kube-api-access-kcptg\") pod \"test-curl-7675-140232305502768\" (UID: \"dfcff708-1cf2-4839-872a-a1300967ad93\") " pod="opendatahub/test-curl-7675-140232305502768" Jul 20 13:28:48.800240 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.800190 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcptg\" (UniqueName: \"kubernetes.io/projected/dfcff708-1cf2-4839-872a-a1300967ad93-kube-api-access-kcptg\") pod \"test-curl-7675-140232305502768\" (UID: \"dfcff708-1cf2-4839-872a-a1300967ad93\") " pod="opendatahub/test-curl-7675-140232305502768" Jul 20 13:28:48.812328 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.812288 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcptg\" (UniqueName: \"kubernetes.io/projected/dfcff708-1cf2-4839-872a-a1300967ad93-kube-api-access-kcptg\") pod \"test-curl-7675-140232305502768\" (UID: \"dfcff708-1cf2-4839-872a-a1300967ad93\") " pod="opendatahub/test-curl-7675-140232305502768" Jul 20 13:28:48.921109 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:48.921000 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232305502768" Jul 20 13:28:49.061942 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:49.061906 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7675-140232305502768"] Jul 20 13:28:49.063953 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:28:49.063920 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfcff708_1cf2_4839_872a_a1300967ad93.slice/crio-15a661f6753ce7c6ab09fe75add97f76cf889eac0e014a132dba216b3860c653 WatchSource:0}: Error finding container 15a661f6753ce7c6ab09fe75add97f76cf889eac0e014a132dba216b3860c653: Status 404 returned error can't find the container with id 15a661f6753ce7c6ab09fe75add97f76cf889eac0e014a132dba216b3860c653 Jul 20 13:28:49.065340 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:49.065322 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 13:28:49.246505 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:49.246404 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7675-140232305502768" event={"ID":"dfcff708-1cf2-4839-872a-a1300967ad93","Type":"ContainerStarted","Data":"15a661f6753ce7c6ab09fe75add97f76cf889eac0e014a132dba216b3860c653"} Jul 20 13:28:50.256672 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:50.256634 2570 generic.go:356] "Generic (PLEG): container finished" podID="dfcff708-1cf2-4839-872a-a1300967ad93" containerID="9ecc0800cd8284e65c116c32a432778ec45e352e9703dec5d8f44c98a1bfe9f7" exitCode=0 Jul 20 13:28:50.257158 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:50.256730 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7675-140232305502768" event={"ID":"dfcff708-1cf2-4839-872a-a1300967ad93","Type":"ContainerDied","Data":"9ecc0800cd8284e65c116c32a432778ec45e352e9703dec5d8f44c98a1bfe9f7"} Jul 20 13:28:51.399946 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.399921 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232305502768" Jul 20 13:28:51.418899 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.418873 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7675-140232305502768_dfcff708-1cf2-4839-872a-a1300967ad93/test-curl-7675-140232305502768/0.log" Jul 20 13:28:51.421118 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.421096 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcptg\" (UniqueName: \"kubernetes.io/projected/dfcff708-1cf2-4839-872a-a1300967ad93-kube-api-access-kcptg\") pod \"dfcff708-1cf2-4839-872a-a1300967ad93\" (UID: \"dfcff708-1cf2-4839-872a-a1300967ad93\") " Jul 20 13:28:51.423261 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.423228 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfcff708-1cf2-4839-872a-a1300967ad93-kube-api-access-kcptg" (OuterVolumeSpecName: "kube-api-access-kcptg") pod "dfcff708-1cf2-4839-872a-a1300967ad93" (UID: "dfcff708-1cf2-4839-872a-a1300967ad93"). InnerVolumeSpecName "kube-api-access-kcptg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:28:51.441881 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.441847 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7675-140232305502768"] Jul 20 13:28:51.445332 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.445306 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7675-140232305502768"] Jul 20 13:28:51.523317 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.523229 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcptg\" (UniqueName: \"kubernetes.io/projected/dfcff708-1cf2-4839-872a-a1300967ad93-kube-api-access-kcptg\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:28:51.576199 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.576164 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7675-140232305503152"] Jul 20 13:28:51.576533 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.576519 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfcff708-1cf2-4839-872a-a1300967ad93" containerName="test-curl-7675-140232305502768" Jul 20 13:28:51.576583 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.576535 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfcff708-1cf2-4839-872a-a1300967ad93" containerName="test-curl-7675-140232305502768" Jul 20 13:28:51.576620 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.576606 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfcff708-1cf2-4839-872a-a1300967ad93" containerName="test-curl-7675-140232305502768" Jul 20 13:28:51.579524 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.579506 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232305503152" Jul 20 13:28:51.587899 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.587870 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7675-140232305503152"] Jul 20 13:28:51.623902 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.623858 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6v9w\" (UniqueName: \"kubernetes.io/projected/b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b-kube-api-access-n6v9w\") pod \"test-curl-7675-140232305503152\" (UID: \"b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b\") " pod="opendatahub/test-curl-7675-140232305503152" Jul 20 13:28:51.724394 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.724326 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6v9w\" (UniqueName: \"kubernetes.io/projected/b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b-kube-api-access-n6v9w\") pod \"test-curl-7675-140232305503152\" (UID: \"b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b\") " pod="opendatahub/test-curl-7675-140232305503152" Jul 20 13:28:51.733332 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.733294 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6v9w\" (UniqueName: \"kubernetes.io/projected/b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b-kube-api-access-n6v9w\") pod \"test-curl-7675-140232305503152\" (UID: \"b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b\") " pod="opendatahub/test-curl-7675-140232305503152" Jul 20 13:28:51.890213 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:51.890181 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232305503152" Jul 20 13:28:52.024512 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:52.024481 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7675-140232305503152"] Jul 20 13:28:52.026484 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:28:52.026459 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8e5f142_e1e5_4bb8_bd9f_5d643223aa9b.slice/crio-ba9314d39748c33fab0133c089fcec675af298c0c0afd07c7355271b4e551c8e WatchSource:0}: Error finding container ba9314d39748c33fab0133c089fcec675af298c0c0afd07c7355271b4e551c8e: Status 404 returned error can't find the container with id ba9314d39748c33fab0133c089fcec675af298c0c0afd07c7355271b4e551c8e Jul 20 13:28:52.268562 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:52.268470 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232305502768" Jul 20 13:28:52.268745 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:52.268470 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15a661f6753ce7c6ab09fe75add97f76cf889eac0e014a132dba216b3860c653" Jul 20 13:28:52.269820 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:52.269792 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7675-140232305503152" event={"ID":"b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b","Type":"ContainerStarted","Data":"ba9314d39748c33fab0133c089fcec675af298c0c0afd07c7355271b4e551c8e"} Jul 20 13:28:52.279499 ip-10-0-141-229 kubenswrapper[2570]: E0720 13:28:52.279464 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7675-140232305502768\" is forbidden: User \"system:node:ip-10-0-141-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-229.ec2.internal' and this object" podUID="dfcff708-1cf2-4839-872a-a1300967ad93" pod="opendatahub/test-curl-7675-140232305502768" Jul 20 13:28:52.521532 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:52.521446 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfcff708-1cf2-4839-872a-a1300967ad93" path="/var/lib/kubelet/pods/dfcff708-1cf2-4839-872a-a1300967ad93/volumes" Jul 20 13:28:53.276031 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:53.275993 2570 generic.go:356] "Generic (PLEG): container finished" podID="b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b" containerID="3c59a52ec9785d607fab558287aabc86f4eccfdc071744f4bc4e397ae7b5a0f7" exitCode=0 Jul 20 13:28:53.276211 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:53.276077 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7675-140232305503152" event={"ID":"b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b","Type":"ContainerDied","Data":"3c59a52ec9785d607fab558287aabc86f4eccfdc071744f4bc4e397ae7b5a0f7"} Jul 20 13:28:54.424863 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.424833 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232305503152" Jul 20 13:28:54.444759 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.444728 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7675-140232305503152_b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b/test-curl-7675-140232305503152/0.log" Jul 20 13:28:54.447827 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.447802 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6v9w\" (UniqueName: \"kubernetes.io/projected/b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b-kube-api-access-n6v9w\") pod \"b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b\" (UID: \"b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b\") " Jul 20 13:28:54.449972 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.449939 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b-kube-api-access-n6v9w" (OuterVolumeSpecName: "kube-api-access-n6v9w") pod "b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b" (UID: "b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b"). InnerVolumeSpecName "kube-api-access-n6v9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:28:54.469947 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.469910 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7675-140232305503152"] Jul 20 13:28:54.472552 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.472520 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7675-140232305503152"] Jul 20 13:28:54.521776 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.521730 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b" path="/var/lib/kubelet/pods/b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b/volumes" Jul 20 13:28:54.548819 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.548722 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6v9w\" (UniqueName: \"kubernetes.io/projected/b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b-kube-api-access-n6v9w\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:28:54.641209 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.641168 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7675-140232302212704"] Jul 20 13:28:54.641549 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.641536 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b" containerName="test-curl-7675-140232305503152" Jul 20 13:28:54.641606 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.641551 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b" containerName="test-curl-7675-140232305503152" Jul 20 13:28:54.641606 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.641604 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8e5f142-e1e5-4bb8-bd9f-5d643223aa9b" containerName="test-curl-7675-140232305503152" Jul 20 13:28:54.644535 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.644515 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:54.651091 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.651057 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7675-140232302212704"] Jul 20 13:28:54.750603 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.750564 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-994q8\" (UniqueName: \"kubernetes.io/projected/5de8056d-cc2d-4f71-bba1-cde722a3b603-kube-api-access-994q8\") pod \"test-curl-7675-140232302212704\" (UID: \"5de8056d-cc2d-4f71-bba1-cde722a3b603\") " pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:54.851746 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.851710 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-994q8\" (UniqueName: \"kubernetes.io/projected/5de8056d-cc2d-4f71-bba1-cde722a3b603-kube-api-access-994q8\") pod \"test-curl-7675-140232302212704\" (UID: \"5de8056d-cc2d-4f71-bba1-cde722a3b603\") " pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:54.863199 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.863166 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-994q8\" (UniqueName: \"kubernetes.io/projected/5de8056d-cc2d-4f71-bba1-cde722a3b603-kube-api-access-994q8\") pod \"test-curl-7675-140232302212704\" (UID: \"5de8056d-cc2d-4f71-bba1-cde722a3b603\") " pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:54.954452 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:54.954405 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:55.100807 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:55.100707 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7675-140232302212704"] Jul 20 13:28:55.103619 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:28:55.103564 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5de8056d_cc2d_4f71_bba1_cde722a3b603.slice/crio-1dc9cd1369125b96fc5396793935ecab64b8b4f3a91c7f0cada504b2412518a1 WatchSource:0}: Error finding container 1dc9cd1369125b96fc5396793935ecab64b8b4f3a91c7f0cada504b2412518a1: Status 404 returned error can't find the container with id 1dc9cd1369125b96fc5396793935ecab64b8b4f3a91c7f0cada504b2412518a1 Jul 20 13:28:55.286897 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:55.286860 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232305503152" Jul 20 13:28:55.287092 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:55.286895 2570 scope.go:117] "RemoveContainer" containerID="3c59a52ec9785d607fab558287aabc86f4eccfdc071744f4bc4e397ae7b5a0f7" Jul 20 13:28:55.288228 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:55.288120 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7675-140232302212704" event={"ID":"5de8056d-cc2d-4f71-bba1-cde722a3b603","Type":"ContainerStarted","Data":"1dc9cd1369125b96fc5396793935ecab64b8b4f3a91c7f0cada504b2412518a1"} Jul 20 13:28:55.789335 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:55.789243 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:28:56.294591 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:56.294550 2570 generic.go:356] "Generic (PLEG): container finished" podID="5de8056d-cc2d-4f71-bba1-cde722a3b603" containerID="2842773dffaa9637be83cb01813cd5b07d479cee4a79e16f5b2588030e60be30" exitCode=0 Jul 20 13:28:56.294800 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:56.294640 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7675-140232302212704" event={"ID":"5de8056d-cc2d-4f71-bba1-cde722a3b603","Type":"ContainerDied","Data":"2842773dffaa9637be83cb01813cd5b07d479cee4a79e16f5b2588030e60be30"} Jul 20 13:28:57.446139 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.446108 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:57.465512 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.465484 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7675-140232302212704_5de8056d-cc2d-4f71-bba1-cde722a3b603/test-curl-7675-140232302212704/0.log" Jul 20 13:28:57.472845 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.472803 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-994q8\" (UniqueName: \"kubernetes.io/projected/5de8056d-cc2d-4f71-bba1-cde722a3b603-kube-api-access-994q8\") pod \"5de8056d-cc2d-4f71-bba1-cde722a3b603\" (UID: \"5de8056d-cc2d-4f71-bba1-cde722a3b603\") " Jul 20 13:28:57.474960 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.474935 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5de8056d-cc2d-4f71-bba1-cde722a3b603-kube-api-access-994q8" (OuterVolumeSpecName: "kube-api-access-994q8") pod "5de8056d-cc2d-4f71-bba1-cde722a3b603" (UID: "5de8056d-cc2d-4f71-bba1-cde722a3b603"). InnerVolumeSpecName "kube-api-access-994q8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:28:57.486911 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.486878 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7675-140232302212704"] Jul 20 13:28:57.489039 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.489009 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7675-140232302212704"] Jul 20 13:28:57.574353 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.574241 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-994q8\" (UniqueName: \"kubernetes.io/projected/5de8056d-cc2d-4f71-bba1-cde722a3b603-kube-api-access-994q8\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:28:57.616224 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.616184 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7675-140232302212704"] Jul 20 13:28:57.616649 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.616628 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5de8056d-cc2d-4f71-bba1-cde722a3b603" containerName="test-curl-7675-140232302212704" Jul 20 13:28:57.616769 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.616651 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5de8056d-cc2d-4f71-bba1-cde722a3b603" containerName="test-curl-7675-140232302212704" Jul 20 13:28:57.616827 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.616774 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5de8056d-cc2d-4f71-bba1-cde722a3b603" containerName="test-curl-7675-140232302212704" Jul 20 13:28:57.619639 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.619616 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:57.626828 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.626790 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7675-140232302212704"] Jul 20 13:28:57.675189 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.675152 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw7r8\" (UniqueName: \"kubernetes.io/projected/e5ef8b87-0d22-488c-adad-1e5faf509570-kube-api-access-rw7r8\") pod \"test-curl-7675-140232302212704\" (UID: \"e5ef8b87-0d22-488c-adad-1e5faf509570\") " pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:57.776437 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.776391 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw7r8\" (UniqueName: \"kubernetes.io/projected/e5ef8b87-0d22-488c-adad-1e5faf509570-kube-api-access-rw7r8\") pod \"test-curl-7675-140232302212704\" (UID: \"e5ef8b87-0d22-488c-adad-1e5faf509570\") " pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:57.784792 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.784757 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw7r8\" (UniqueName: \"kubernetes.io/projected/e5ef8b87-0d22-488c-adad-1e5faf509570-kube-api-access-rw7r8\") pod \"test-curl-7675-140232302212704\" (UID: \"e5ef8b87-0d22-488c-adad-1e5faf509570\") " pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:57.929816 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:57.929779 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:58.273499 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:58.273307 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7675-140232302212704"] Jul 20 13:28:58.276095 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:28:58.276066 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5ef8b87_0d22_488c_adad_1e5faf509570.slice/crio-098ee6ff97549b6165839de0e0bcd8b885d9195f5bb8877b3d1f647bec64710a WatchSource:0}: Error finding container 098ee6ff97549b6165839de0e0bcd8b885d9195f5bb8877b3d1f647bec64710a: Status 404 returned error can't find the container with id 098ee6ff97549b6165839de0e0bcd8b885d9195f5bb8877b3d1f647bec64710a Jul 20 13:28:58.305662 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:58.305626 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:28:58.305662 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:58.305645 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1dc9cd1369125b96fc5396793935ecab64b8b4f3a91c7f0cada504b2412518a1" Jul 20 13:28:58.306903 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:58.306873 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7675-140232302212704" event={"ID":"e5ef8b87-0d22-488c-adad-1e5faf509570","Type":"ContainerStarted","Data":"098ee6ff97549b6165839de0e0bcd8b885d9195f5bb8877b3d1f647bec64710a"} Jul 20 13:28:58.317401 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:58.317356 2570 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7675-140232302212704" oldPodUID="5de8056d-cc2d-4f71-bba1-cde722a3b603" podUID="e5ef8b87-0d22-488c-adad-1e5faf509570" Jul 20 13:28:58.521759 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:58.521723 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5de8056d-cc2d-4f71-bba1-cde722a3b603" path="/var/lib/kubelet/pods/5de8056d-cc2d-4f71-bba1-cde722a3b603/volumes" Jul 20 13:28:59.312824 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:59.312786 2570 generic.go:356] "Generic (PLEG): container finished" podID="e5ef8b87-0d22-488c-adad-1e5faf509570" containerID="694165cfb8a2811fe56dd6143bcbc294bf7acccb73a655e32809d38ad8144f6e" exitCode=0 Jul 20 13:28:59.313020 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:28:59.312845 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7675-140232302212704" event={"ID":"e5ef8b87-0d22-488c-adad-1e5faf509570","Type":"ContainerDied","Data":"694165cfb8a2811fe56dd6143bcbc294bf7acccb73a655e32809d38ad8144f6e"} Jul 20 13:29:00.461206 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.461176 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:29:00.481310 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.481278 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7675-140232302212704_e5ef8b87-0d22-488c-adad-1e5faf509570/test-curl-7675-140232302212704/0.log" Jul 20 13:29:00.504445 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.504414 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw7r8\" (UniqueName: \"kubernetes.io/projected/e5ef8b87-0d22-488c-adad-1e5faf509570-kube-api-access-rw7r8\") pod \"e5ef8b87-0d22-488c-adad-1e5faf509570\" (UID: \"e5ef8b87-0d22-488c-adad-1e5faf509570\") " Jul 20 13:29:00.506273 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.506245 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7675-140232302212704"] Jul 20 13:29:00.506570 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.506544 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ef8b87-0d22-488c-adad-1e5faf509570-kube-api-access-rw7r8" (OuterVolumeSpecName: "kube-api-access-rw7r8") pod "e5ef8b87-0d22-488c-adad-1e5faf509570" (UID: "e5ef8b87-0d22-488c-adad-1e5faf509570"). InnerVolumeSpecName "kube-api-access-rw7r8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:29:00.510142 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.510120 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7675-140232302212704"] Jul 20 13:29:00.521337 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.521304 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5ef8b87-0d22-488c-adad-1e5faf509570" path="/var/lib/kubelet/pods/e5ef8b87-0d22-488c-adad-1e5faf509570/volumes" Jul 20 13:29:00.605346 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.605315 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw7r8\" (UniqueName: \"kubernetes.io/projected/e5ef8b87-0d22-488c-adad-1e5faf509570-kube-api-access-rw7r8\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:29:00.669286 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.669245 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7675-140232302213712"] Jul 20 13:29:00.669652 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.669636 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ef8b87-0d22-488c-adad-1e5faf509570" containerName="test-curl-7675-140232302212704" Jul 20 13:29:00.669695 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.669654 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ef8b87-0d22-488c-adad-1e5faf509570" containerName="test-curl-7675-140232302212704" Jul 20 13:29:00.669731 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.669711 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ef8b87-0d22-488c-adad-1e5faf509570" containerName="test-curl-7675-140232302212704" Jul 20 13:29:00.672944 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.672924 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302213712" Jul 20 13:29:00.682935 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.682904 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7675-140232302213712"] Jul 20 13:29:00.706498 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.706453 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzlmr\" (UniqueName: \"kubernetes.io/projected/6c0ca2da-fe93-44b2-b80e-d68d973fa842-kube-api-access-jzlmr\") pod \"test-curl-7675-140232302213712\" (UID: \"6c0ca2da-fe93-44b2-b80e-d68d973fa842\") " pod="opendatahub/test-curl-7675-140232302213712" Jul 20 13:29:00.807636 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.807599 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzlmr\" (UniqueName: \"kubernetes.io/projected/6c0ca2da-fe93-44b2-b80e-d68d973fa842-kube-api-access-jzlmr\") pod \"test-curl-7675-140232302213712\" (UID: \"6c0ca2da-fe93-44b2-b80e-d68d973fa842\") " pod="opendatahub/test-curl-7675-140232302213712" Jul 20 13:29:00.821777 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.821756 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzlmr\" (UniqueName: \"kubernetes.io/projected/6c0ca2da-fe93-44b2-b80e-d68d973fa842-kube-api-access-jzlmr\") pod \"test-curl-7675-140232302213712\" (UID: \"6c0ca2da-fe93-44b2-b80e-d68d973fa842\") " pod="opendatahub/test-curl-7675-140232302213712" Jul 20 13:29:00.983482 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:00.983354 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302213712" Jul 20 13:29:01.152876 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:01.152847 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7675-140232302213712"] Jul 20 13:29:01.155242 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:29:01.155212 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c0ca2da_fe93_44b2_b80e_d68d973fa842.slice/crio-162754255a36e21701593bda313b2952f3e5689cbe05e95c333ed032e4168940 WatchSource:0}: Error finding container 162754255a36e21701593bda313b2952f3e5689cbe05e95c333ed032e4168940: Status 404 returned error can't find the container with id 162754255a36e21701593bda313b2952f3e5689cbe05e95c333ed032e4168940 Jul 20 13:29:01.327429 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:01.327323 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302212704" Jul 20 13:29:01.327429 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:01.327350 2570 scope.go:117] "RemoveContainer" containerID="694165cfb8a2811fe56dd6143bcbc294bf7acccb73a655e32809d38ad8144f6e" Jul 20 13:29:01.328733 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:01.328523 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7675-140232302213712" event={"ID":"6c0ca2da-fe93-44b2-b80e-d68d973fa842","Type":"ContainerStarted","Data":"162754255a36e21701593bda313b2952f3e5689cbe05e95c333ed032e4168940"} Jul 20 13:29:02.335685 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:02.335647 2570 generic.go:356] "Generic (PLEG): container finished" podID="6c0ca2da-fe93-44b2-b80e-d68d973fa842" containerID="8867f81675bbd670f54a8855f4755a137909b95905024574c7f9fb273f6ab635" exitCode=0 Jul 20 13:29:02.336085 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:02.335699 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7675-140232302213712" event={"ID":"6c0ca2da-fe93-44b2-b80e-d68d973fa842","Type":"ContainerDied","Data":"8867f81675bbd670f54a8855f4755a137909b95905024574c7f9fb273f6ab635"} Jul 20 13:29:03.474995 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:03.474968 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302213712" Jul 20 13:29:03.495958 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:03.495932 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7675-140232302213712_6c0ca2da-fe93-44b2-b80e-d68d973fa842/test-curl-7675-140232302213712/0.log" Jul 20 13:29:03.518828 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:03.518797 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7675-140232302213712"] Jul 20 13:29:03.522920 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:03.522892 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7675-140232302213712"] Jul 20 13:29:03.532146 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:03.532120 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzlmr\" (UniqueName: \"kubernetes.io/projected/6c0ca2da-fe93-44b2-b80e-d68d973fa842-kube-api-access-jzlmr\") pod \"6c0ca2da-fe93-44b2-b80e-d68d973fa842\" (UID: \"6c0ca2da-fe93-44b2-b80e-d68d973fa842\") " Jul 20 13:29:03.534231 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:03.534207 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c0ca2da-fe93-44b2-b80e-d68d973fa842-kube-api-access-jzlmr" (OuterVolumeSpecName: "kube-api-access-jzlmr") pod "6c0ca2da-fe93-44b2-b80e-d68d973fa842" (UID: "6c0ca2da-fe93-44b2-b80e-d68d973fa842"). InnerVolumeSpecName "kube-api-access-jzlmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:29:03.633009 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:03.632919 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzlmr\" (UniqueName: \"kubernetes.io/projected/6c0ca2da-fe93-44b2-b80e-d68d973fa842-kube-api-access-jzlmr\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:29:04.347952 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:04.347928 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="162754255a36e21701593bda313b2952f3e5689cbe05e95c333ed032e4168940" Jul 20 13:29:04.347952 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:04.347949 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7675-140232302213712" Jul 20 13:29:04.357396 ip-10-0-141-229 kubenswrapper[2570]: E0720 13:29:04.357347 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7675-140232302213712\" is forbidden: User \"system:node:ip-10-0-141-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-229.ec2.internal' and this object" podUID="6c0ca2da-fe93-44b2-b80e-d68d973fa842" pod="opendatahub/test-curl-7675-140232302213712" Jul 20 13:29:04.521496 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:04.521465 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c0ca2da-fe93-44b2-b80e-d68d973fa842" path="/var/lib/kubelet/pods/6c0ca2da-fe93-44b2-b80e-d68d973fa842/volumes" Jul 20 13:29:06.468417 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.468357 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:06.740075 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.739989 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q"] Jul 20 13:29:06.740363 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.740350 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c0ca2da-fe93-44b2-b80e-d68d973fa842" containerName="test-curl-7675-140232302213712" Jul 20 13:29:06.740363 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.740376 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c0ca2da-fe93-44b2-b80e-d68d973fa842" containerName="test-curl-7675-140232302213712" Jul 20 13:29:06.740469 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.740436 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c0ca2da-fe93-44b2-b80e-d68d973fa842" containerName="test-curl-7675-140232302213712" Jul 20 13:29:06.743587 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.743565 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q" Jul 20 13:29:06.746255 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.746234 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 20 13:29:06.746383 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.746340 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4s4l\"" Jul 20 13:29:06.753156 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.753121 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q"] Jul 20 13:29:06.860209 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.860164 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/943a2b28-aecb-4a90-ad9a-83fbe2e46615-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q\" (UID: \"943a2b28-aecb-4a90-ad9a-83fbe2e46615\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q" Jul 20 13:29:06.961033 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.960993 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/943a2b28-aecb-4a90-ad9a-83fbe2e46615-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q\" (UID: \"943a2b28-aecb-4a90-ad9a-83fbe2e46615\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q" Jul 20 13:29:06.961792 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:06.961766 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/943a2b28-aecb-4a90-ad9a-83fbe2e46615-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q\" (UID: \"943a2b28-aecb-4a90-ad9a-83fbe2e46615\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q" Jul 20 13:29:07.055090 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:07.054998 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q" Jul 20 13:29:07.195487 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:07.195452 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q"] Jul 20 13:29:07.198921 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:29:07.198889 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943a2b28_aecb_4a90_ad9a_83fbe2e46615.slice/crio-2355c34447a1bd2497a01caa20015699c4e37e9c9c7888e87270f200cecc15f5 WatchSource:0}: Error finding container 2355c34447a1bd2497a01caa20015699c4e37e9c9c7888e87270f200cecc15f5: Status 404 returned error can't find the container with id 2355c34447a1bd2497a01caa20015699c4e37e9c9c7888e87270f200cecc15f5 Jul 20 13:29:07.348184 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:07.348141 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:07.363730 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:07.363696 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q" event={"ID":"943a2b28-aecb-4a90-ad9a-83fbe2e46615","Type":"ContainerStarted","Data":"2355c34447a1bd2497a01caa20015699c4e37e9c9c7888e87270f200cecc15f5"} Jul 20 13:29:07.551626 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:07.551598 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:08.370664 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:08.370629 2570 generic.go:356] "Generic (PLEG): container finished" podID="943a2b28-aecb-4a90-ad9a-83fbe2e46615" containerID="b18520041dc6bbcea86767e41ca5e1ba7e01c36db0a63fde877dd6292479c944" exitCode=0 Jul 20 13:29:08.370863 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:08.370702 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q" event={"ID":"943a2b28-aecb-4a90-ad9a-83fbe2e46615","Type":"ContainerDied","Data":"b18520041dc6bbcea86767e41ca5e1ba7e01c36db0a63fde877dd6292479c944"} Jul 20 13:29:09.514697 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:09.514668 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q" Jul 20 13:29:09.584925 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:09.584890 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/943a2b28-aecb-4a90-ad9a-83fbe2e46615-maas-api-service-ca\") pod \"943a2b28-aecb-4a90-ad9a-83fbe2e46615\" (UID: \"943a2b28-aecb-4a90-ad9a-83fbe2e46615\") " Jul 20 13:29:09.585239 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:09.585217 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/943a2b28-aecb-4a90-ad9a-83fbe2e46615-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "943a2b28-aecb-4a90-ad9a-83fbe2e46615" (UID: "943a2b28-aecb-4a90-ad9a-83fbe2e46615"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:29:09.686511 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:09.686420 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/943a2b28-aecb-4a90-ad9a-83fbe2e46615-maas-api-service-ca\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:29:10.381788 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:10.381718 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q" Jul 20 13:29:10.381965 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:10.381715 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q" event={"ID":"943a2b28-aecb-4a90-ad9a-83fbe2e46615","Type":"ContainerDied","Data":"2355c34447a1bd2497a01caa20015699c4e37e9c9c7888e87270f200cecc15f5"} Jul 20 13:29:10.381965 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:10.381852 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2355c34447a1bd2497a01caa20015699c4e37e9c9c7888e87270f200cecc15f5" Jul 20 13:29:17.422769 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:17.422725 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:18.705719 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:18.705690 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:29:18.718246 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:18.718219 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:29:21.817518 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:21.817476 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:21.834156 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:21.834115 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q"] Jul 20 13:29:21.835850 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:21.835824 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7f57a3-w478q"] Jul 20 13:29:21.877141 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:21.877107 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:22.213634 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:22.213599 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:22.521579 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:22.521498 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="943a2b28-aecb-4a90-ad9a-83fbe2e46615" path="/var/lib/kubelet/pods/943a2b28-aecb-4a90-ad9a-83fbe2e46615/volumes" Jul 20 13:29:28.051236 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.051204 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:28.114963 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.114933 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:28.356114 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.356079 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:28.470183 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.470148 2570 generic.go:356] "Generic (PLEG): container finished" podID="2b40e4b5-77e1-47be-8d27-aa605ac2b87f" containerID="237dbc4541a115949e9de70faf2424fb7e686031ca015298e8c1ebbf00fcc712" exitCode=0 Jul 20 13:29:28.470390 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.470211 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" event={"ID":"2b40e4b5-77e1-47be-8d27-aa605ac2b87f","Type":"ContainerDied","Data":"237dbc4541a115949e9de70faf2424fb7e686031ca015298e8c1ebbf00fcc712"} Jul 20 13:29:28.470599 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.470585 2570 scope.go:117] "RemoveContainer" containerID="237dbc4541a115949e9de70faf2424fb7e686031ca015298e8c1ebbf00fcc712" Jul 20 13:29:28.574405 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.574294 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:28.685453 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.685409 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql"] Jul 20 13:29:28.685857 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.685841 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="943a2b28-aecb-4a90-ad9a-83fbe2e46615" containerName="revoke-keys" Jul 20 13:29:28.685911 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.685859 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="943a2b28-aecb-4a90-ad9a-83fbe2e46615" containerName="revoke-keys" Jul 20 13:29:28.685949 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.685929 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="943a2b28-aecb-4a90-ad9a-83fbe2e46615" containerName="revoke-keys" Jul 20 13:29:28.688974 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.688900 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" Jul 20 13:29:28.691051 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.691025 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 20 13:29:28.691179 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.691062 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4s4l\"" Jul 20 13:29:28.697540 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.697505 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql"] Jul 20 13:29:28.869685 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.869634 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eca4548-ba22-4814-833e-886b7382031b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql\" (UID: \"0eca4548-ba22-4814-833e-886b7382031b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" Jul 20 13:29:28.970282 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.970241 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eca4548-ba22-4814-833e-886b7382031b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql\" (UID: \"0eca4548-ba22-4814-833e-886b7382031b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" Jul 20 13:29:28.970951 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:28.970928 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eca4548-ba22-4814-833e-886b7382031b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql\" (UID: \"0eca4548-ba22-4814-833e-886b7382031b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" Jul 20 13:29:29.000714 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:29.000673 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" Jul 20 13:29:29.153774 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:29.153740 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql"] Jul 20 13:29:29.156146 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:29:29.156099 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eca4548_ba22_4814_833e_886b7382031b.slice/crio-8db8f90b183115a8dd80e18c695c7cff5aef1d388b6ee5fc217d4634a99b026e WatchSource:0}: Error finding container 8db8f90b183115a8dd80e18c695c7cff5aef1d388b6ee5fc217d4634a99b026e: Status 404 returned error can't find the container with id 8db8f90b183115a8dd80e18c695c7cff5aef1d388b6ee5fc217d4634a99b026e Jul 20 13:29:29.383145 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:29.383114 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:29.477621 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:29.477580 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" event={"ID":"2b40e4b5-77e1-47be-8d27-aa605ac2b87f","Type":"ContainerStarted","Data":"4fc0f3ac3ea64aec2992dde33cdf238b7b38b3483bd72fbb86105bb840c87c1b"} Jul 20 13:29:29.477846 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:29.477820 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:29:29.479126 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:29.479099 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" event={"ID":"0eca4548-ba22-4814-833e-886b7382031b","Type":"ContainerStarted","Data":"667a9c6580467d0f8057a63e1c6ae17dd645b5aef559ad53790a4081dbc24deb"} Jul 20 13:29:29.479248 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:29.479146 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" event={"ID":"0eca4548-ba22-4814-833e-886b7382031b","Type":"ContainerStarted","Data":"8db8f90b183115a8dd80e18c695c7cff5aef1d388b6ee5fc217d4634a99b026e"} Jul 20 13:29:29.483938 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:29.483912 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:29.516565 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:29.516486 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" podStartSLOduration=1.516466874 podStartE2EDuration="1.516466874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:29:29.515969309 +0000 UTC m=+2711.488175139" watchObservedRunningTime="2026-07-20 13:29:29.516466874 +0000 UTC m=+2711.488672705" Jul 20 13:29:30.485771 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:30.485737 2570 generic.go:356] "Generic (PLEG): container finished" podID="0eca4548-ba22-4814-833e-886b7382031b" containerID="667a9c6580467d0f8057a63e1c6ae17dd645b5aef559ad53790a4081dbc24deb" exitCode=0 Jul 20 13:29:30.486196 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:30.485812 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" event={"ID":"0eca4548-ba22-4814-833e-886b7382031b","Type":"ContainerDied","Data":"667a9c6580467d0f8057a63e1c6ae17dd645b5aef559ad53790a4081dbc24deb"} Jul 20 13:29:31.630647 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:31.630616 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" Jul 20 13:29:31.796603 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:31.796494 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eca4548-ba22-4814-833e-886b7382031b-maas-api-service-ca\") pod \"0eca4548-ba22-4814-833e-886b7382031b\" (UID: \"0eca4548-ba22-4814-833e-886b7382031b\") " Jul 20 13:29:31.796912 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:31.796891 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eca4548-ba22-4814-833e-886b7382031b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0eca4548-ba22-4814-833e-886b7382031b" (UID: "0eca4548-ba22-4814-833e-886b7382031b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:29:31.897897 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:31.897859 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eca4548-ba22-4814-833e-886b7382031b-maas-api-service-ca\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\"" Jul 20 13:29:32.502584 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:32.502547 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" Jul 20 13:29:32.502792 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:32.502575 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql" event={"ID":"0eca4548-ba22-4814-833e-886b7382031b","Type":"ContainerDied","Data":"8db8f90b183115a8dd80e18c695c7cff5aef1d388b6ee5fc217d4634a99b026e"} Jul 20 13:29:32.502792 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:32.502667 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8db8f90b183115a8dd80e18c695c7cff5aef1d388b6ee5fc217d4634a99b026e" Jul 20 13:29:39.351601 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:39.351563 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:39.930177 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:39.930141 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:40.488841 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:40.488800 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l5dmb" Jul 20 13:29:43.777134 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:43.777082 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:43.784027 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:43.783988 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql"] Jul 20 13:29:43.791540 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:43.791507 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b8383c-t9bql"] Jul 20 13:29:44.165603 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:44.165574 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:44.526785 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:44.526683 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eca4548-ba22-4814-833e-886b7382031b" path="/var/lib/kubelet/pods/0eca4548-ba22-4814-833e-886b7382031b/volumes" Jul 20 13:29:45.772893 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:45.772856 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:50.104029 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:50.103989 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:50.203774 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:50.203739 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:50.290199 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:50.290164 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5cvxg"] Jul 20 13:29:50.375640 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:50.375553 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:50.652050 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:50.651947 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:57.666252 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:57.666214 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749978ccd7-kpss7_7bfa49f3-e344-4930-8b09-0edb79e8c08b/manager/0.log" Jul 20 13:29:57.672944 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:57.672922 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749978ccd7-kpss7_7bfa49f3-e344-4930-8b09-0edb79e8c08b/copy-manifests/0.log" Jul 20 13:29:58.402054 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:58.402021 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742555-t7lf2_5c9155cb-766c-42b5-a4d9-9c61d8fdee20/cleanup/0.log" Jul 20 13:29:58.857696 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:58.857647 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:58.912208 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:58.912161 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:29:59.894726 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:29:59.894680 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-x9pvp_66326b8e-f834-450e-8d10-13ace27f33ed/manager/0.log" Jul 20 13:30:00.013691 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:00.013652 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qppk4_9cb81595-2f69-4f83-bfc2-2e3ff63d5223/manager/0.log" Jul 20 13:30:00.131540 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:00.131508 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7zn8l_fc0684d0-174c-4ec6-9bb4-f4947db32687/kuadrant-console-plugin/0.log" Jul 20 13:30:00.249792 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:00.249693 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-t86cr_62d8932f-d2d1-4fcd-9f2b-8308ab8174e7/registry-server/0.log" Jul 20 13:30:00.372613 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:00.372581 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-l5dmb_2b40e4b5-77e1-47be-8d27-aa605ac2b87f/manager/1.log" Jul 20 13:30:00.482885 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:00.482854 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5cvxg_e394caee-8e24-427a-b423-0d9fa708ea22/limitador/0.log" Jul 20 13:30:00.957106 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:00.957080 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf482zmw_01a555a6-3312-4a51-a13c-2fad0b03fe1a/istio-proxy/0.log" Jul 20 13:30:01.837628 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:01.837592 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-594bbb7854-524rq_e66b1363-7be7-4cad-ad86-053fbe8c5596/router/0.log" Jul 20 13:30:02.779664 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:02.779632 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-kml9d_34eb34d8-b3f3-4d8e-b0b8-698606098fb7/main/0.log" Jul 20 13:30:08.001737 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:08.001700 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:30:08.100189 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:08.100143 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" limit=1024 Jul 20 13:30:11.056700 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:11.056667 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wvc67_1220141b-d9f7-4e87-95e8-cc658b486b7f/global-pull-secret-syncer/0.log" Jul 20 13:30:11.101940 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:11.101911 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-55hwm_36c5fc1a-f0e9-4443-b922-cd84c06112e5/konnectivity-agent/0.log" Jul 20 13:30:11.221842 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:11.221813 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-229.ec2.internal_192fe6abc0214ce0c06c38f944481f45/haproxy/0.log" Jul 20 13:30:16.559649 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:16.559553 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-x9pvp_66326b8e-f834-450e-8d10-13ace27f33ed/manager/0.log" Jul 20 13:30:16.589687 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:16.589656 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qppk4_9cb81595-2f69-4f83-bfc2-2e3ff63d5223/manager/0.log" Jul 20 13:30:16.615065 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:16.615039 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7zn8l_fc0684d0-174c-4ec6-9bb4-f4947db32687/kuadrant-console-plugin/0.log" Jul 20 13:30:16.654624 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:16.654591 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-t86cr_62d8932f-d2d1-4fcd-9f2b-8308ab8174e7/registry-server/0.log" Jul 20 13:30:16.684498 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:16.684467 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-l5dmb_2b40e4b5-77e1-47be-8d27-aa605ac2b87f/manager/1.log" Jul 20 13:30:16.836175 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:16.836142 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-l5dmb_2b40e4b5-77e1-47be-8d27-aa605ac2b87f/manager/0.log" Jul 20 13:30:16.875242 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:16.875201 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5cvxg_e394caee-8e24-427a-b423-0d9fa708ea22/limitador/0.log" Jul 20 13:30:19.054867 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:19.054835 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-spg2n_a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d/node-exporter/0.log" Jul 20 13:30:19.072946 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:19.072913 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-spg2n_a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d/kube-rbac-proxy/0.log" Jul 20 13:30:19.091378 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:19.091348 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-spg2n_a48f2efa-fdc7-41b7-a4e3-1a7d5ddc8a5d/init-textfile/0.log" Jul 20 13:30:19.189669 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:19.189633 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9fa8696b-a1e9-4ed5-bd40-02f7adcef946/prometheus/0.log" Jul 20 13:30:19.207892 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:19.207873 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9fa8696b-a1e9-4ed5-bd40-02f7adcef946/config-reloader/0.log" Jul 20 13:30:19.226654 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:19.226579 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9fa8696b-a1e9-4ed5-bd40-02f7adcef946/thanos-sidecar/0.log" Jul 20 13:30:19.246254 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:19.246228 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9fa8696b-a1e9-4ed5-bd40-02f7adcef946/kube-rbac-proxy-web/0.log" Jul 20 13:30:19.266168 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:19.266146 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9fa8696b-a1e9-4ed5-bd40-02f7adcef946/kube-rbac-proxy/0.log" Jul 20 13:30:19.286919 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:19.286886 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9fa8696b-a1e9-4ed5-bd40-02f7adcef946/kube-rbac-proxy-thanos/0.log" Jul 20 13:30:19.308006 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:19.307984 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9fa8696b-a1e9-4ed5-bd40-02f7adcef946/init-config-reloader/0.log" Jul 20 13:30:20.715457 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:20.715427 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-xfvcw_a7ce0ee4-eaab-4666-87fa-4cb351d17e56/networking-console-plugin/0.log" Jul 20 13:30:21.338258 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.338229 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/1.log" Jul 20 13:30:21.344684 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.344653 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-cwqb4_071ec3e8-c145-4cb7-9ab5-116975ec402c/console-operator/2.log" Jul 20 13:30:21.361733 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.361699 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb"] Jul 20 13:30:21.362059 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.362047 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eca4548-ba22-4814-833e-886b7382031b" containerName="revoke-keys" Jul 20 13:30:21.362103 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.362061 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eca4548-ba22-4814-833e-886b7382031b" containerName="revoke-keys" Jul 20 13:30:21.362138 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.362121 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eca4548-ba22-4814-833e-886b7382031b" containerName="revoke-keys" Jul 20 13:30:21.366818 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.366792 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.369663 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.369642 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-478p7\"/\"default-dockercfg-dd2k8\"" Jul 20 13:30:21.369855 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.369842 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-478p7\"/\"kube-root-ca.crt\"" Jul 20 13:30:21.370007 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.369993 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-478p7\"/\"openshift-service-ca.crt\"" Jul 20 13:30:21.375826 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.375796 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb"] Jul 20 13:30:21.545549 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.545515 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-podres\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.545744 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.545564 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-lib-modules\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.545744 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.545604 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-proc\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.545744 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.545697 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-sys\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.545883 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.545746 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srkqc\" (UniqueName: \"kubernetes.io/projected/ad238594-6b84-4ae6-8d6d-a69301024d87-kube-api-access-srkqc\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.646841 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.646719 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-podres\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.646841 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.646780 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-lib-modules\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.646841 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.646806 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-proc\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.646841 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.646830 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-sys\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.647300 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.646876 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srkqc\" (UniqueName: \"kubernetes.io/projected/ad238594-6b84-4ae6-8d6d-a69301024d87-kube-api-access-srkqc\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.647300 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.646927 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-podres\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.647300 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.646942 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-sys\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.647300 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.646926 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-lib-modules\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.647300 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.646985 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad238594-6b84-4ae6-8d6d-a69301024d87-proc\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.655387 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.655338 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srkqc\" (UniqueName: \"kubernetes.io/projected/ad238594-6b84-4ae6-8d6d-a69301024d87-kube-api-access-srkqc\") pod \"perf-node-gather-daemonset-bp2cb\" (UID: \"ad238594-6b84-4ae6-8d6d-a69301024d87\") " pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.678644 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.678573 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:21.825029 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.825001 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb"] Jul 20 13:30:21.827760 ip-10-0-141-229 kubenswrapper[2570]: W0720 13:30:21.827738 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podad238594_6b84_4ae6_8d6d_a69301024d87.slice/crio-af575402c163f5ac656978921bce9306bfdebdca5e63bb708caaaa2ebf78bcdd WatchSource:0}: Error finding container af575402c163f5ac656978921bce9306bfdebdca5e63bb708caaaa2ebf78bcdd: Status 404 returned error can't find the container with id af575402c163f5ac656978921bce9306bfdebdca5e63bb708caaaa2ebf78bcdd Jul 20 13:30:21.997224 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:21.997191 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-hpsh8_615ac5be-f68e-4381-a35a-5ce6ac9df15b/download-server/0.log" Jul 20 13:30:22.610321 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:22.610288 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-rqn64_72ab29c8-e484-4ffa-b85b-48848df8d63c/volume-data-source-validator/0.log" Jul 20 13:30:22.745599 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:22.745566 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" event={"ID":"ad238594-6b84-4ae6-8d6d-a69301024d87","Type":"ContainerStarted","Data":"e787c5baeb2502ff9f3c5dced1de8ea680100677599e06cedcaa4d119faad4e4"} Jul 20 13:30:22.745806 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:22.745622 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" event={"ID":"ad238594-6b84-4ae6-8d6d-a69301024d87","Type":"ContainerStarted","Data":"af575402c163f5ac656978921bce9306bfdebdca5e63bb708caaaa2ebf78bcdd"} Jul 20 13:30:22.745806 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:22.745645 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:22.761435 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:22.761362 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" podStartSLOduration=1.761344399 podStartE2EDuration="1.761344399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:30:22.760211725 +0000 UTC m=+2764.732417556" watchObservedRunningTime="2026-07-20 13:30:22.761344399 +0000 UTC m=+2764.733550243" Jul 20 13:30:23.638871 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:23.638841 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tlzx5_8319b26b-2857-4f07-aeae-9116dbb18e10/dns/0.log" Jul 20 13:30:23.657187 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:23.657162 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tlzx5_8319b26b-2857-4f07-aeae-9116dbb18e10/kube-rbac-proxy/0.log" Jul 20 13:30:23.684062 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:23.684028 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jstg9_c1fc1557-eab5-4747-9378-59a27b43a32a/dns-node-resolver/0.log" Jul 20 13:30:24.226268 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:24.226226 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8f5ccc466-5ksdn_b23c4f77-5615-4e72-ad77-2f228693db38/registry/0.log" Jul 20 13:30:24.285529 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:24.285501 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vs9qv_fd720144-9924-46ef-a880-0c196604451b/node-ca/0.log" Jul 20 13:30:25.243691 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:25.243661 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf482zmw_01a555a6-3312-4a51-a13c-2fad0b03fe1a/istio-proxy/0.log" Jul 20 13:30:25.703535 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:25.703501 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-594bbb7854-524rq_e66b1363-7be7-4cad-ad86-053fbe8c5596/router/0.log" Jul 20 13:30:26.396698 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:26.396670 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8zhcc_f09b4592-c85d-4f65-b6d1-63e0d175afa4/serve-healthcheck-canary/0.log" Jul 20 13:30:26.960318 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:26.960289 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8gb2_cb1dfcff-aa2f-477e-ab35-302441b374c3/kube-rbac-proxy/0.log" Jul 20 13:30:26.978791 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:26.978757 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8gb2_cb1dfcff-aa2f-477e-ab35-302441b374c3/exporter/0.log" Jul 20 13:30:26.998669 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:26.998640 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8gb2_cb1dfcff-aa2f-477e-ab35-302441b374c3/extractor/0.log" Jul 20 13:30:28.760495 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:28.760465 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-478p7/perf-node-gather-daemonset-bp2cb" Jul 20 13:30:29.836216 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:29.836185 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749978ccd7-kpss7_7bfa49f3-e344-4930-8b09-0edb79e8c08b/manager/0.log" Jul 20 13:30:29.853606 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:29.853579 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749978ccd7-kpss7_7bfa49f3-e344-4930-8b09-0edb79e8c08b/copy-manifests/0.log" Jul 20 13:30:31.046923 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:31.046891 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf482zmw_01a555a6-3312-4a51-a13c-2fad0b03fe1a/istio-proxy/0.log" Jul 20 13:30:31.240252 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:31.240218 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-586f8df4c4-jz65n_a7055925-99db-41ec-a730-5e633ce939b2/manager/0.log" Jul 20 13:30:31.426824 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:31.426793 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-594bbb7854-524rq_e66b1363-7be7-4cad-ad86-053fbe8c5596/router/0.log" Jul 20 13:30:32.753990 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:32.753904 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf482zmw_01a555a6-3312-4a51-a13c-2fad0b03fe1a/istio-proxy/0.log" Jul 20 13:30:33.142834 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:33.142805 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-594bbb7854-524rq_e66b1363-7be7-4cad-ad86-053fbe8c5596/router/0.log" Jul 20 13:30:36.608088 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:36.608060 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-zwnsq_ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca/kube-storage-version-migrator-operator/1.log" Jul 20 13:30:36.608587 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:36.608558 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-zwnsq_ab4d1035-f1b8-440d-bb30-a0ab1d8ea0ca/kube-storage-version-migrator-operator/0.log" Jul 20 13:30:37.871912 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:37.871876 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhm8h_fc1cf019-6b91-49b4-b73a-1c41a257269d/kube-multus-additional-cni-plugins/0.log" Jul 20 13:30:37.891119 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:37.891091 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhm8h_fc1cf019-6b91-49b4-b73a-1c41a257269d/egress-router-binary-copy/0.log" Jul 20 13:30:37.910434 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:37.910406 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhm8h_fc1cf019-6b91-49b4-b73a-1c41a257269d/cni-plugins/0.log" Jul 20 13:30:37.929675 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:37.929647 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhm8h_fc1cf019-6b91-49b4-b73a-1c41a257269d/bond-cni-plugin/0.log" Jul 20 13:30:37.950478 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:37.950451 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhm8h_fc1cf019-6b91-49b4-b73a-1c41a257269d/routeoverride-cni/0.log" Jul 20 13:30:37.969823 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:37.969791 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhm8h_fc1cf019-6b91-49b4-b73a-1c41a257269d/whereabouts-cni-bincopy/0.log" Jul 20 13:30:37.988543 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:37.988517 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhm8h_fc1cf019-6b91-49b4-b73a-1c41a257269d/whereabouts-cni/0.log" Jul 20 13:30:38.019123 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:38.019091 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-kpsm8_717fdc62-4efb-490e-bd34-e147ac34ddbf/kube-multus/0.log" Jul 20 13:30:38.117011 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:38.116984 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6gkrz_24ad9090-71b2-4f16-9389-19f115738cf6/network-metrics-daemon/0.log" Jul 20 13:30:38.135996 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:38.135889 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6gkrz_24ad9090-71b2-4f16-9389-19f115738cf6/kube-rbac-proxy/0.log" Jul 20 13:30:39.504986 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:39.504948 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z95dr_9eb5f9c7-9a60-4075-b022-e5308b67d79c/ovn-controller/0.log" Jul 20 13:30:39.521315 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:39.521289 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z95dr_9eb5f9c7-9a60-4075-b022-e5308b67d79c/ovn-acl-logging/0.log" Jul 20 13:30:39.542198 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:39.542164 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z95dr_9eb5f9c7-9a60-4075-b022-e5308b67d79c/kube-rbac-proxy-node/0.log" Jul 20 13:30:39.560169 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:39.560133 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z95dr_9eb5f9c7-9a60-4075-b022-e5308b67d79c/kube-rbac-proxy-ovn-metrics/0.log" Jul 20 13:30:39.578216 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:39.578190 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z95dr_9eb5f9c7-9a60-4075-b022-e5308b67d79c/northd/0.log" Jul 20 13:30:39.597409 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:39.597383 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z95dr_9eb5f9c7-9a60-4075-b022-e5308b67d79c/nbdb/0.log" Jul 20 13:30:39.619568 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:39.619536 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z95dr_9eb5f9c7-9a60-4075-b022-e5308b67d79c/sbdb/0.log" Jul 20 13:30:39.674714 ip-10-0-141-229 kubenswrapper[2570]: I0720 13:30:39.674673 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z95dr_9eb5f9c7-9a60-4075-b022-e5308b67d79c/ovnkube-controller/0.log"