Jul 02 22:59:45.454058 ip-10-0-136-71 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 02 22:59:45.454072 ip-10-0-136-71 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 02 22:59:45.454081 ip-10-0-136-71 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 02 22:59:45.454311 ip-10-0-136-71 systemd[1]: Failed to start Kubernetes Kubelet. Jul 02 22:59:55.660449 ip-10-0-136-71 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 02 22:59:55.660470 ip-10-0-136-71 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 85d7265b7d054bdea0f55f4b5ce74c1d -- Jul 02 23:01:31.731840 ip-10-0-136-71 systemd[1]: Starting Kubernetes Kubelet... Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.154968 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160617 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160627 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160631 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160636 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160639 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160642 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160645 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:01:32.201328 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160648 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160651 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160654 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160657 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160660 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160662 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160665 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160668 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160671 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160673 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160676 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160678 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160681 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160684 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160686 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160690 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160692 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160695 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160698 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160701 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:01:32.464182 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160703 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:01:32.234202 ip-10-0-136-71 systemd[1]: Started Kubernetes Kubelet. Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160706 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160708 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160711 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160714 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160717 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160719 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160722 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160726 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160728 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160732 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160735 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160737 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160740 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160743 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160745 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160748 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160751 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160753 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160756 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:01:32.476748 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160758 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160761 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160763 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160766 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160769 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160771 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160776 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160781 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160784 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160788 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160791 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160794 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160797 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160801 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160804 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160807 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160810 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160813 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160817 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:01:32.478408 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160819 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160822 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160825 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160828 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160831 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160834 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160838 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160841 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160844 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160847 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160850 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160852 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160855 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160857 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160860 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160863 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160865 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160868 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160871 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160873 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:01:32.479274 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160876 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160879 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160882 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160886 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160889 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160892 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160895 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160898 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160901 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160903 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160906 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160910 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.160913 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161612 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161619 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161622 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161625 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161628 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161637 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161640 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:01:32.481411 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161643 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161645 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161648 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161652 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161657 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161660 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161663 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161666 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161669 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161672 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161675 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161678 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161680 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161685 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161687 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161690 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161692 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161695 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161698 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161700 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:01:32.483557 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161703 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161706 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161708 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161712 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161715 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161718 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161720 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161725 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161729 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161732 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161735 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161744 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161747 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161750 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161753 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161756 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161759 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161762 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161765 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:01:32.485978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161767 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161770 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161772 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161775 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161778 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161780 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161783 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161788 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161791 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161794 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161797 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161800 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161803 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161806 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161809 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161812 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161817 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161820 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161823 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:01:32.488351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161825 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161828 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161830 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161833 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161836 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161839 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161847 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161851 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161854 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161856 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161859 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161861 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161864 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161866 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161869 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161872 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161874 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161877 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161879 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161882 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:01:32.490186 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161885 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161887 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161890 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161892 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161895 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161897 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161900 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161903 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161905 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161909 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161911 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161914 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161917 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.161919 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162274 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162279 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162281 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162285 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162290 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162293 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:01:32.491037 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162295 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162298 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162301 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162304 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162307 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162310 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162312 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162315 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162318 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162322 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162326 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162329 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162332 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162335 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162338 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162341 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162344 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162347 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162350 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162353 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:01:32.491845 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162355 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162360 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162362 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162365 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162368 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162370 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162373 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162378 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162380 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162383 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162386 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162389 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162391 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162394 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162397 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162399 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162402 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162405 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162408 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162410 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:01:32.493002 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162413 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162416 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162418 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162421 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162425 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162429 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162433 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162436 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162439 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162442 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162445 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162448 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162451 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162455 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162458 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162461 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162465 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162467 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:01:32.493923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162470 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162474 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162477 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162479 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162482 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162485 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162488 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162491 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162493 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162496 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162498 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162501 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162504 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162506 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162509 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162511 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162514 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162517 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162519 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162522 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:01:32.494553 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162525 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162527 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162530 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162532 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162535 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162537 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162540 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162544 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162546 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162549 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162552 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162554 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162557 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162560 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.162563 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162641 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162648 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162653 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162657 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162662 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162666 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 23:01:32.495161 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162670 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162674 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162678 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162681 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162684 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162687 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162690 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162694 2568 flags.go:64] FLAG: --cgroup-root="" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162697 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162700 2568 flags.go:64] FLAG: --client-ca-file="" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162702 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162705 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162710 2568 flags.go:64] FLAG: --cluster-domain="" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162713 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162716 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162719 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162723 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162726 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162730 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162733 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162736 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162739 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162742 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162745 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 23:01:32.495948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162750 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162753 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162756 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162759 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162762 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162765 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162768 2568 flags.go:64] FLAG: --enable-server="true" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162771 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162776 2568 flags.go:64] FLAG: --event-burst="100" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162779 2568 flags.go:64] FLAG: --event-qps="50" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162782 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162785 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162788 2568 flags.go:64] FLAG: --eviction-hard="" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162791 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162794 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162797 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162800 2568 flags.go:64] FLAG: --eviction-soft="" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162803 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162806 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162809 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162812 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162815 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162818 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162821 2568 flags.go:64] FLAG: --feature-gates="" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162827 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 23:01:32.496739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162830 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162835 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162838 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162841 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162844 2568 flags.go:64] FLAG: --help="false" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162847 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162850 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162855 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162858 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162861 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162864 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162867 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162870 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162873 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162876 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162879 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162882 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162885 2568 flags.go:64] FLAG: --kube-reserved="" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162888 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162891 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162894 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162897 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162899 2568 flags.go:64] FLAG: --lock-file="" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162902 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 23:01:32.497497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162905 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162908 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162913 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162916 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162918 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162921 2568 flags.go:64] FLAG: --logging-format="text" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162924 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162927 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162930 2568 flags.go:64] FLAG: --manifest-url="" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162934 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162938 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162941 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162945 2568 flags.go:64] FLAG: --max-pods="110" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162948 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162951 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162955 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162958 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162961 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162964 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162967 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162973 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162976 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162979 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 23:01:32.498165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162982 2568 flags.go:64] FLAG: --pod-cidr="" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162986 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162991 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162994 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.162997 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163000 2568 flags.go:64] FLAG: --port="10250" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163003 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163006 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a87b904da66aba65" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163009 2568 flags.go:64] FLAG: --qos-reserved="" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163012 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163015 2568 flags.go:64] FLAG: --register-node="true" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163018 2568 flags.go:64] FLAG: --register-with-taints="" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163022 2568 flags.go:64] FLAG: --registry-burst="10" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163025 2568 flags.go:64] FLAG: --registry-qps="5" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163027 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163030 2568 flags.go:64] FLAG: --reserved-memory="" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163034 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163036 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163039 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163043 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163046 2568 flags.go:64] FLAG: --runonce="false" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163049 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163052 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163055 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163059 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 23:01:32.498858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163063 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163066 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163069 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163072 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163074 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163077 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163080 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163083 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163086 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163089 2568 flags.go:64] FLAG: --system-cgroups="" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163092 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163097 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163100 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163103 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163107 2568 flags.go:64] FLAG: --tls-min-version="" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163110 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163112 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163128 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163132 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163136 2568 flags.go:64] FLAG: --v="2" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163140 2568 flags.go:64] FLAG: --version="false" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163144 2568 flags.go:64] FLAG: --vmodule="" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163148 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163151 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163279 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:01:32.499838 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163283 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163288 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163291 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163294 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163297 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163300 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163305 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163308 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163311 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163313 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163316 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163319 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163322 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163324 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163327 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163330 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163333 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163336 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163339 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163341 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:01:32.500520 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163344 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163347 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163349 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163352 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163354 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163359 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163362 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163365 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163368 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163371 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163374 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163377 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163380 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163386 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163390 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163393 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163396 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163399 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163403 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:01:32.629171 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163406 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163409 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163413 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163415 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163419 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163421 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163424 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163427 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163430 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163433 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163436 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163439 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163442 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163444 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163447 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163449 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163452 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163455 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163457 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163460 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:01:32.629806 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163462 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163465 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163468 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163470 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163473 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163475 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163481 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163484 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163488 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163491 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163494 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163498 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163501 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163504 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163507 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163509 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163512 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163515 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163517 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:01:32.630329 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163520 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163523 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163526 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163528 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163531 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163534 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163537 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163539 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163542 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163545 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163547 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163550 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163552 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163555 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163558 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163561 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163563 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163566 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163568 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:01:32.630868 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.163572 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.163592 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.169781 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.169799 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.169827 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.169835 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169906 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169912 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169916 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169920 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169924 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169928 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169931 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169936 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:01:32.631351 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169940 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169943 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169948 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169951 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169955 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169959 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169963 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169967 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169971 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169975 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169978 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169982 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169986 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169990 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169994 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.169998 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170001 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170005 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170009 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170013 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:01:32.631799 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170017 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170022 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170026 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170030 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170034 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170039 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170044 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170048 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170052 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170056 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170061 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170064 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170071 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170077 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170081 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170084 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170088 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170093 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170097 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:01:32.632326 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170103 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170108 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170113 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170117 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170122 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170126 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170132 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170137 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170142 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170146 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170150 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170154 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170158 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170161 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170164 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170167 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170170 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170172 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170175 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:01:32.632884 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170178 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170180 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170183 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170186 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170188 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170191 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170195 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170199 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170202 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170205 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170207 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170210 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170212 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170215 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170218 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170220 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170223 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170226 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170228 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170231 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:01:32.633391 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170233 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170236 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170238 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170241 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170244 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170246 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170249 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170252 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170254 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170257 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170260 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170264 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170268 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.170275 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170424 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:01:32.633960 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170429 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170434 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170438 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170442 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170449 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170453 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170457 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170461 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170465 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170469 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170473 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170476 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170480 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170484 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170488 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170491 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170495 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170499 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170503 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170506 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:01:32.634369 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170510 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170514 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170517 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170522 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170526 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170530 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170533 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170576 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170792 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170798 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170802 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170809 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170815 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170820 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170825 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170830 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170837 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170842 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170847 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:01:32.634942 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170850 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170860 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170865 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170869 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170873 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170877 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170881 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170886 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170890 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170894 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170898 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170902 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170906 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170910 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170918 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170922 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170926 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170930 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170934 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170938 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:01:32.635433 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170942 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170956 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170961 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170965 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170969 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170975 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170987 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170993 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.170998 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171007 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171014 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171019 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171024 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171028 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171033 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171037 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171041 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171046 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171055 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:01:32.635991 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171059 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171063 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171067 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171071 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171075 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171080 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171084 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171088 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171091 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171095 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171099 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171103 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171111 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171115 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171119 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171124 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171128 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171133 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171137 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:01:32.636507 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:32.171141 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.171149 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.171353 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.174065 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.175477 2568 server.go:1019] "Starting client certificate rotation" Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.175570 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.175625 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.178564 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.180341 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.191345 2568 log.go:25] "Validated CRI v1 runtime API" Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.193679 2568 log.go:25] "Validated CRI v1 image API" Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.194551 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 23:01:32.637074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.201492 2568 fs.go:135] Filesystem UUIDs: map[3935927c-615b-46bd-9cdb-ae7b6911885c:/dev/nvme0n1p3 450de082-b791-402e-bddc-6b4ec95cc2ac:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 02 23:01:32.637426 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.201506 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 23:01:32.637426 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.205064 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.209751 2568 manager.go:216] Machine: {Timestamp:2026-07-02 23:01:32.208322881 +0000 UTC m=+0.361560532 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091157 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d5c254186b377f009f4a11ae4ba22 SystemUUID:ec2d5c25-4186-b377-f009-f4a11ae4ba22 BootID:85d7265b-7d05-4bde-a0f5-5f4b5ce74c1d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2b:6e:d2:dc:4b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2b:6e:d2:dc:4b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:12:27:1c:44:f8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.209852 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.210254 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.213245 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.213418 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-71.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.214310 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.214321 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.214329 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.215035 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.216724 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.220849 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.220991 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.221018 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.221031 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.221044 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.221055 2568 kubelet.go:398] "Adding apiserver pod source" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.221066 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 23:01:32.637639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.221390 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.221409 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.224832 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.227024 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.227050 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228917 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228930 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228936 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228942 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228950 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228955 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228965 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228970 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228977 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228982 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228990 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.228995 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.230552 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.230560 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.233395 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-71.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.233416 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-71.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.233469 2568 server.go:1268] "Started kubelet" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.233528 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.233547 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.233552 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.233651 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.233746 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zp568" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.235477 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.235528 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.238657 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.241335 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zp568" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.242052 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-71.ec2.internal.18be9b5516451331 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-71.ec2.internal,UID:ip-10-0-136-71.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-71.ec2.internal,},FirstTimestamp:2026-07-02 23:01:32.233446193 +0000 UTC m=+0.386683848,LastTimestamp:2026-07-02 23:01:32.233446193 +0000 UTC m=+0.386683848,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-71.ec2.internal,}" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.243196 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.243210 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.243480 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.244024 2568 factory.go:55] Registering systemd factory Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.244049 2568 factory.go:223] Registration of the systemd container factory successfully Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.244247 2568 factory.go:153] Registering CRI-O factory Jul 02 23:01:32.638183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.244260 2568 factory.go:223] Registration of the crio container factory successfully Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.244304 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.244326 2568 factory.go:103] Registering Raw factory Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.244339 2568 manager.go:1196] Started watching for new ooms in manager Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.244735 2568 manager.go:349] Starting recovery of all containers Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.244964 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.244966 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.244989 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.245125 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.245133 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.246386 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-71.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.246479 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.254882 2568 manager.go:354] Recovery completed Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.259111 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.260718 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.260744 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.260758 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.261204 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.261209 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.261213 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.261230 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.261882 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.261904 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.261914 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.263275 2568 policy_none.go:49] "None policy: Start" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.263290 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.263301 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.265143 2568 policy_none.go:47] "Start" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.341775 2568 manager.go:337] "Starting Device Plugin manager" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.341921 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.341933 2568 server.go:87] "Starting device plugin registration server" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.342132 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.342142 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.342245 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.342323 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.342332 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.342799 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.342847 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.380698 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.381804 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.381834 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.381851 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.381879 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.384329 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.443217 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.443873 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.443900 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.443911 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:01:32.639492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.443931 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.454526 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.454544 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-71.ec2.internal\": node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.469182 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.482380 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-71.ec2.internal"] Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.482449 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.483966 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.483997 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.484011 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.486261 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.486360 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.486390 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.487044 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.487073 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.487082 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.487098 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.487120 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.487134 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.490032 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.490058 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.490780 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.490826 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.490851 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.535620 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-71.ec2.internal\" not found" node="ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.541965 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-71.ec2.internal\" not found" node="ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.546955 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/168cb0651600516a255a42a3464e8a1f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal\" (UID: \"168cb0651600516a255a42a3464e8a1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.546984 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/934cc245ae5a1b254602b20dfff22029-config\") pod \"kube-apiserver-proxy-ip-10-0-136-71.ec2.internal\" (UID: \"934cc245ae5a1b254602b20dfff22029\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.547006 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/168cb0651600516a255a42a3464e8a1f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal\" (UID: \"168cb0651600516a255a42a3464e8a1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.569623 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.647637 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/168cb0651600516a255a42a3464e8a1f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal\" (UID: \"168cb0651600516a255a42a3464e8a1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.804676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.647657 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/168cb0651600516a255a42a3464e8a1f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal\" (UID: \"168cb0651600516a255a42a3464e8a1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.806224 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.647672 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/934cc245ae5a1b254602b20dfff22029-config\") pod \"kube-apiserver-proxy-ip-10-0-136-71.ec2.internal\" (UID: \"934cc245ae5a1b254602b20dfff22029\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.806224 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.647697 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/934cc245ae5a1b254602b20dfff22029-config\") pod \"kube-apiserver-proxy-ip-10-0-136-71.ec2.internal\" (UID: \"934cc245ae5a1b254602b20dfff22029\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.806224 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.647727 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/168cb0651600516a255a42a3464e8a1f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal\" (UID: \"168cb0651600516a255a42a3464e8a1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.806224 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.647766 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/168cb0651600516a255a42a3464e8a1f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal\" (UID: \"168cb0651600516a255a42a3464e8a1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.806224 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.669684 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:32.806224 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.770526 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:32.982299 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.837688 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.982299 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:32.845371 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-71.ec2.internal" Jul 02 23:01:32.982299 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.871472 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:32.982299 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:32.971920 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:33.147621 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:33.072833 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:33.275794 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:33.173473 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:33.275794 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:33.174557 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 23:01:33.275794 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:33.174676 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 23:01:33.275794 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:33.248791 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 22:56:33 +0000 UTC" deadline="2028-02-05 15:46:06.332811644 +0000 UTC" Jul 02 23:01:33.275794 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:33.248812 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13984h44m33.08400288s" Jul 02 23:01:33.275794 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:33.273738 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:34.526146 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:34.454732 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:34.656628 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:34.557504 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-71.ec2.internal\" not found" Jul 02 23:01:34.656628 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:34.601444 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:01:34.656628 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:34.610849 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod168cb0651600516a255a42a3464e8a1f.slice/crio-7cfebff019e758d91c760f7051e3d2044af1c3675e308ccfaa9a87c3bf8eb414 WatchSource:0}: Error finding container 7cfebff019e758d91c760f7051e3d2044af1c3675e308ccfaa9a87c3bf8eb414: Status 404 returned error can't find the container with id 7cfebff019e758d91c760f7051e3d2044af1c3675e308ccfaa9a87c3bf8eb414 Jul 02 23:01:34.656628 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:34.618361 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:01:34.656628 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:34.624258 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-71.ec2.internal" Jul 02 23:01:34.656628 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:34.634835 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:01:34.656628 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:34.636507 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" Jul 02 23:01:34.656628 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:34.646325 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:01:34.852693 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:34.744633 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:01:35.016528 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:34.868533 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.302251 2568 apiserver.go:52] "Watching apiserver" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.310593 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.312611 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-hzb5r","openshift-network-operator/iptables-alerter-hxllf","openshift-ovn-kubernetes/ovnkube-node-qmwh5","kube-system/konnectivity-agent-77nbd","kube-system/kube-apiserver-proxy-ip-10-0-136-71.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal","openshift-multus/multus-additional-cni-plugins-xr5tz","openshift-multus/multus-k5dh5","kube-system/global-pull-secret-syncer-lnsbx","openshift-cluster-node-tuning-operator/tuned-zf25z","openshift-dns/node-resolver-4vks2","openshift-image-registry/node-ca-gfdss","openshift-multus/network-metrics-daemon-mm52v"] Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.315121 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.315190 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.317297 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.319540 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.319708 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.319913 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pvwxj\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.321051 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.321810 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.323010 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.323327 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.323482 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xbbbq\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.323519 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.323481 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.323542 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.324025 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.324085 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.324541 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.324586 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.324737 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.324773 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vnjrj\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.326317 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.327049 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.328628 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.330851 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gfdss" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.335305 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hxllf" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.337198 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.337341 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:35.348056 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.337415 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.337711 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kn928\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.337793 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.337930 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.338019 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.338027 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.338178 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.338264 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.338388 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nwnf2\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.338442 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.338558 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.338676 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c26sq\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.338708 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.338857 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.338901 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w8htz\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.339152 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.340015 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.340466 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.342793 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4vks2" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.342820 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.342902 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9qjm6\"" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343122 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-etc-openvswitch\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-run-ovn\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343191 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-node-log\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343241 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1dac54d6-06f0-4f56-b1c8-6e24e59ea5af-agent-certs\") pod \"konnectivity-agent-77nbd\" (UID: \"1dac54d6-06f0-4f56-b1c8-6e24e59ea5af\") " pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343273 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-systemd\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343297 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-lib-modules\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343319 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-host\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.359440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343349 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-cni-netd\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343377 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343401 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-kubernetes\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343423 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-sysctl-d\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-run\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-slash\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343487 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343487 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-run-openvswitch\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343532 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-log-socket\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343558 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/17b1942c-8c3b-4510-bee8-32742e97493a-env-overrides\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343600 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh29p\" (UniqueName: \"kubernetes.io/projected/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-kube-api-access-bh29p\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343665 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bb3efb95-e804-4cae-b3b5-220db506cfe8-kubelet-config\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343698 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z4qk\" (UniqueName: \"kubernetes.io/projected/8ddcad19-8642-411d-8a79-f815899d94ee-kube-api-access-6z4qk\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343733 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/17b1942c-8c3b-4510-bee8-32742e97493a-ovnkube-script-lib\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343760 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ddcad19-8642-411d-8a79-f815899d94ee-etc-tuned\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343780 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343819 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-run-systemd\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.361171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343933 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.343966 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/17b1942c-8c3b-4510-bee8-32742e97493a-ovnkube-config\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344010 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txzx7\" (UniqueName: \"kubernetes.io/projected/bad930f0-e377-445a-88a5-7e81b222ce9b-kube-api-access-txzx7\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344036 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-socket-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344060 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-kubelet\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344081 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-cni-bin\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344103 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2ckk\" (UniqueName: \"kubernetes.io/projected/17b1942c-8c3b-4510-bee8-32742e97493a-kube-api-access-l2ckk\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-registration-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344174 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-etc-selinux\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-sys-fs\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344219 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bb3efb95-e804-4cae-b3b5-220db506cfe8-dbus\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344262 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344286 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-systemd-units\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344310 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344333 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1dac54d6-06f0-4f56-b1c8-6e24e59ea5af-konnectivity-ca\") pod \"konnectivity-agent-77nbd\" (UID: \"1dac54d6-06f0-4f56-b1c8-6e24e59ea5af\") " pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344366 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-modprobe-d\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.362264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344397 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-sysconfig\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-run-netns\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-var-lib-openvswitch\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/17b1942c-8c3b-4510-bee8-32742e97493a-ovn-node-metrics-cert\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344488 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-device-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344509 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-sysctl-conf\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344529 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-sys\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344552 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-var-lib-kubelet\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.344603 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ddcad19-8642-411d-8a79-f815899d94ee-tmp\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.345276 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.345485 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.345808 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g6rzb\"" Jul 02 23:01:35.364240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.351247 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:01:35.429429 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.380129 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xltns" Jul 02 23:01:35.429429 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.385300 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xltns" Jul 02 23:01:35.429429 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.426331 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 23:01:35.446035 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.445979 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.446035 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446015 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-sys-fs\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.446202 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446038 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ddcad19-8642-411d-8a79-f815899d94ee-tmp\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.446202 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446067 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-run-multus-certs\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.446202 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446098 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ce7afefc-665b-46ab-9895-38facdf8467f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.446202 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446117 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-cni-bin\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.446202 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-etc-selinux\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.446202 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446163 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-system-cni-dir\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.446202 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446189 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1dac54d6-06f0-4f56-b1c8-6e24e59ea5af-konnectivity-ca\") pod \"konnectivity-agent-77nbd\" (UID: \"1dac54d6-06f0-4f56-b1c8-6e24e59ea5af\") " pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-modprobe-d\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446238 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-os-release\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446264 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f933ee1-d50b-4391-9881-d002c7ef4ab9-cni-binary-copy\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446289 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/17b1942c-8c3b-4510-bee8-32742e97493a-ovn-node-metrics-cert\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-sysctl-conf\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446332 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-sys\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446352 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-cnibin\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446370 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-socket-dir-parent\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-etc-openvswitch\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-run-ovn\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446423 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-systemd\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446437 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-lib-modules\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446469 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-var-lib-cni-multus\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:35.446515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446515 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-kubernetes\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446537 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-run\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446561 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-run-netns\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446593 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd-tmp-dir\") pod \"node-resolver-4vks2\" (UID: \"12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd\") " pod="openshift-dns/node-resolver-4vks2" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446620 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh29p\" (UniqueName: \"kubernetes.io/projected/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-kube-api-access-bh29p\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446640 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bb3efb95-e804-4cae-b3b5-220db506cfe8-kubelet-config\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446659 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-conf-dir\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446683 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-system-cni-dir\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446706 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/17b1942c-8c3b-4510-bee8-32742e97493a-ovnkube-script-lib\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446744 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ddcad19-8642-411d-8a79-f815899d94ee-etc-tuned\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446769 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-run-k8s-cni-cncf-io\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446805 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-hostroot\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446825 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed9789e7-960c-4bb6-b65d-fd51ae63a29e-host\") pod \"node-ca-gfdss\" (UID: \"ed9789e7-960c-4bb6-b65d-fd51ae63a29e\") " pod="openshift-image-registry/node-ca-gfdss" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446847 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9w4j\" (UniqueName: \"kubernetes.io/projected/ed9789e7-960c-4bb6-b65d-fd51ae63a29e-kube-api-access-s9w4j\") pod \"node-ca-gfdss\" (UID: \"ed9789e7-960c-4bb6-b65d-fd51ae63a29e\") " pod="openshift-image-registry/node-ca-gfdss" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446861 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446870 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ce7afefc-665b-46ab-9895-38facdf8467f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446899 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-cni-dir\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.447309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446912 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-sys-fs\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446925 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446957 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/17b1942c-8c3b-4510-bee8-32742e97493a-ovnkube-config\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.446985 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txzx7\" (UniqueName: \"kubernetes.io/projected/bad930f0-e377-445a-88a5-7e81b222ce9b-kube-api-access-txzx7\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447009 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-socket-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447035 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n6k5\" (UniqueName: \"kubernetes.io/projected/9f933ee1-d50b-4391-9881-d002c7ef4ab9-kube-api-access-7n6k5\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447059 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4cv4\" (UniqueName: \"kubernetes.io/projected/f8dce277-b4c5-440c-8304-fdcbabbce375-kube-api-access-h4cv4\") pod \"iptables-alerter-hxllf\" (UID: \"f8dce277-b4c5-440c-8304-fdcbabbce375\") " pod="openshift-network-operator/iptables-alerter-hxllf" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447103 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-kubelet\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447206 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447218 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-cni-bin\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447259 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bb3efb95-e804-4cae-b3b5-220db506cfe8-kubelet-config\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447276 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-modprobe-d\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447319 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-sys\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447325 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-etc-selinux\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447425 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-sysctl-conf\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447429 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-kubelet\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447482 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.448109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447524 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-socket-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447536 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-systemd\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447584 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-etc-openvswitch\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.447664 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.447740 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs podName:bad930f0-e377-445a-88a5-7e81b222ce9b nodeName:}" failed. No retries permitted until 2026-07-02 23:01:35.947706318 +0000 UTC m=+3.020958669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs") pod "network-metrics-daemon-mm52v" (UID: "bad930f0-e377-445a-88a5-7e81b222ce9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447891 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-run-ovn\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447898 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1dac54d6-06f0-4f56-b1c8-6e24e59ea5af-konnectivity-ca\") pod \"konnectivity-agent-77nbd\" (UID: \"1dac54d6-06f0-4f56-b1c8-6e24e59ea5af\") " pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447905 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/17b1942c-8c3b-4510-bee8-32742e97493a-ovnkube-script-lib\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.447950 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-kubernetes\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448008 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2ckk\" (UniqueName: \"kubernetes.io/projected/17b1942c-8c3b-4510-bee8-32742e97493a-kube-api-access-l2ckk\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448011 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-run\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448055 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-registration-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448080 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bb3efb95-e804-4cae-b3b5-220db506cfe8-dbus\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448127 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448161 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-sysconfig\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448190 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqtqc\" (UniqueName: \"kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc\") pod \"network-check-target-hzb5r\" (UID: \"cc68e25f-5e72-4518-9f6e-b7b1870bdf97\") " pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448260 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-os-release\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.448927 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448295 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/17b1942c-8c3b-4510-bee8-32742e97493a-ovnkube-config\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448304 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-systemd-units\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448359 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bb3efb95-e804-4cae-b3b5-220db506cfe8-dbus\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448368 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-cnibin\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.448377 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448421 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-systemd-units\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.448432 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret podName:bb3efb95-e804-4cae-b3b5-220db506cfe8 nodeName:}" failed. No retries permitted until 2026-07-02 23:01:35.948409658 +0000 UTC m=+3.021662010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret") pod "global-pull-secret-syncer-lnsbx" (UID: "bb3efb95-e804-4cae-b3b5-220db506cfe8") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448434 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-sysconfig\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448474 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgxsh\" (UniqueName: \"kubernetes.io/projected/ce7afefc-665b-46ab-9895-38facdf8467f-kube-api-access-wgxsh\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448479 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-registration-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448503 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqndp\" (UniqueName: \"kubernetes.io/projected/12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd-kube-api-access-gqndp\") pod \"node-resolver-4vks2\" (UID: \"12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd\") " pod="openshift-dns/node-resolver-4vks2" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448526 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-run-netns\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448539 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448582 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-run-netns\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448593 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-lib-modules\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.449792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.448717 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-var-lib-openvswitch\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.451638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.449148 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-var-lib-openvswitch\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.451749 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.451675 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-device-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.451749 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.451721 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-var-lib-kubelet\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.451821 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.451755 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8dce277-b4c5-440c-8304-fdcbabbce375-host-slash\") pod \"iptables-alerter-hxllf\" (UID: \"f8dce277-b4c5-440c-8304-fdcbabbce375\") " pod="openshift-network-operator/iptables-alerter-hxllf" Jul 02 23:01:35.451821 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.451806 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-node-log\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.451981 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.451844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1dac54d6-06f0-4f56-b1c8-6e24e59ea5af-agent-certs\") pod \"konnectivity-agent-77nbd\" (UID: \"1dac54d6-06f0-4f56-b1c8-6e24e59ea5af\") " pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:01:35.451981 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.451861 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ddcad19-8642-411d-8a79-f815899d94ee-etc-tuned\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.451981 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.451868 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-host\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.451981 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.451921 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-var-lib-kubelet\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.451981 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.451953 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8dce277-b4c5-440c-8304-fdcbabbce375-iptables-alerter-script\") pod \"iptables-alerter-hxllf\" (UID: \"f8dce277-b4c5-440c-8304-fdcbabbce375\") " pod="openshift-network-operator/iptables-alerter-hxllf" Jul 02 23:01:35.451981 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.451973 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-device-dir\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.452264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452045 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-var-lib-kubelet\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.452264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452046 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd-hosts-file\") pod \"node-resolver-4vks2\" (UID: \"12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd\") " pod="openshift-dns/node-resolver-4vks2" Jul 02 23:01:35.452264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452100 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-node-log\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.452264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452135 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-cni-netd\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.452264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452176 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-sysctl-d\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.452264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452226 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-etc-kubernetes\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.452529 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452265 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce7afefc-665b-46ab-9895-38facdf8467f-cni-binary-copy\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.452529 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452301 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-slash\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.452529 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452345 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-run-openvswitch\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.452529 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-log-socket\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.452529 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452439 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/17b1942c-8c3b-4510-bee8-32742e97493a-env-overrides\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.452529 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452490 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z4qk\" (UniqueName: \"kubernetes.io/projected/8ddcad19-8642-411d-8a79-f815899d94ee-kube-api-access-6z4qk\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.452529 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452525 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-var-lib-cni-bin\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.452885 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452635 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-daemon-config\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.452885 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452697 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.452885 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452732 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed9789e7-960c-4bb6-b65d-fd51ae63a29e-serviceca\") pod \"node-ca-gfdss\" (UID: \"ed9789e7-960c-4bb6-b65d-fd51ae63a29e\") " pod="openshift-image-registry/node-ca-gfdss" Jul 02 23:01:35.452885 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452770 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-run-systemd\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.453118 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452903 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-log-socket\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.453118 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.452977 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-run-systemd\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.453118 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.453048 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-cni-netd\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.453267 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.453144 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-etc-sysctl-d\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.453267 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.453194 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-host-slash\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.453267 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.453259 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17b1942c-8c3b-4510-bee8-32742e97493a-run-openvswitch\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.453410 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.453273 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/17b1942c-8c3b-4510-bee8-32742e97493a-env-overrides\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.453410 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.453336 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ddcad19-8642-411d-8a79-f815899d94ee-host\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.454898 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.454859 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1dac54d6-06f0-4f56-b1c8-6e24e59ea5af-agent-certs\") pod \"konnectivity-agent-77nbd\" (UID: \"1dac54d6-06f0-4f56-b1c8-6e24e59ea5af\") " pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:01:35.455523 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.455492 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ddcad19-8642-411d-8a79-f815899d94ee-tmp\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.455829 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.455762 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/17b1942c-8c3b-4510-bee8-32742e97493a-ovn-node-metrics-cert\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.463615 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.463531 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2ckk\" (UniqueName: \"kubernetes.io/projected/17b1942c-8c3b-4510-bee8-32742e97493a-kube-api-access-l2ckk\") pod \"ovnkube-node-qmwh5\" (UID: \"17b1942c-8c3b-4510-bee8-32742e97493a\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.463798 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.463775 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txzx7\" (UniqueName: \"kubernetes.io/projected/bad930f0-e377-445a-88a5-7e81b222ce9b-kube-api-access-txzx7\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:35.465677 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.465467 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh29p\" (UniqueName: \"kubernetes.io/projected/23cd16b1-4932-4ef3-b5ad-e1d4a48650d7-kube-api-access-bh29p\") pod \"aws-ebs-csi-driver-node-dwgmf\" (UID: \"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.467329 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.467306 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z4qk\" (UniqueName: \"kubernetes.io/projected/8ddcad19-8642-411d-8a79-f815899d94ee-kube-api-access-6z4qk\") pod \"tuned-zf25z\" (UID: \"8ddcad19-8642-411d-8a79-f815899d94ee\") " pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.467520 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.467450 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-71.ec2.internal" event={"ID":"934cc245ae5a1b254602b20dfff22029","Type":"ContainerStarted","Data":"4443642420f4fde069c18e933b47d972a30409933fd92440ca32ede7504cf832"} Jul 02 23:01:35.468939 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.468914 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" event={"ID":"168cb0651600516a255a42a3464e8a1f","Type":"ContainerStarted","Data":"7cfebff019e758d91c760f7051e3d2044af1c3675e308ccfaa9a87c3bf8eb414"} Jul 02 23:01:35.553495 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553469 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-conf-dir\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.553495 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553420 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-conf-dir\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.553688 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553523 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-system-cni-dir\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.553688 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553558 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-run-k8s-cni-cncf-io\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.553688 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553612 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-hostroot\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.553688 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553634 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed9789e7-960c-4bb6-b65d-fd51ae63a29e-host\") pod \"node-ca-gfdss\" (UID: \"ed9789e7-960c-4bb6-b65d-fd51ae63a29e\") " pod="openshift-image-registry/node-ca-gfdss" Jul 02 23:01:35.553688 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553662 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9w4j\" (UniqueName: \"kubernetes.io/projected/ed9789e7-960c-4bb6-b65d-fd51ae63a29e-kube-api-access-s9w4j\") pod \"node-ca-gfdss\" (UID: \"ed9789e7-960c-4bb6-b65d-fd51ae63a29e\") " pod="openshift-image-registry/node-ca-gfdss" Jul 02 23:01:35.553688 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553685 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ce7afefc-665b-46ab-9895-38facdf8467f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.553961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553710 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-cni-dir\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.553961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n6k5\" (UniqueName: \"kubernetes.io/projected/9f933ee1-d50b-4391-9881-d002c7ef4ab9-kube-api-access-7n6k5\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.553961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553760 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4cv4\" (UniqueName: \"kubernetes.io/projected/f8dce277-b4c5-440c-8304-fdcbabbce375-kube-api-access-h4cv4\") pod \"iptables-alerter-hxllf\" (UID: \"f8dce277-b4c5-440c-8304-fdcbabbce375\") " pod="openshift-network-operator/iptables-alerter-hxllf" Jul 02 23:01:35.553961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553798 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqtqc\" (UniqueName: \"kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc\") pod \"network-check-target-hzb5r\" (UID: \"cc68e25f-5e72-4518-9f6e-b7b1870bdf97\") " pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:35.553961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553841 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-os-release\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.553961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553864 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-cnibin\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.553961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553898 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgxsh\" (UniqueName: \"kubernetes.io/projected/ce7afefc-665b-46ab-9895-38facdf8467f-kube-api-access-wgxsh\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.553961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553931 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqndp\" (UniqueName: \"kubernetes.io/projected/12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd-kube-api-access-gqndp\") pod \"node-resolver-4vks2\" (UID: \"12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd\") " pod="openshift-dns/node-resolver-4vks2" Jul 02 23:01:35.553961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8dce277-b4c5-440c-8304-fdcbabbce375-host-slash\") pod \"iptables-alerter-hxllf\" (UID: \"f8dce277-b4c5-440c-8304-fdcbabbce375\") " pod="openshift-network-operator/iptables-alerter-hxllf" Jul 02 23:01:35.554303 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.553997 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-var-lib-kubelet\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.554303 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554211 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-cnibin\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.554462 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554334 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed9789e7-960c-4bb6-b65d-fd51ae63a29e-host\") pod \"node-ca-gfdss\" (UID: \"ed9789e7-960c-4bb6-b65d-fd51ae63a29e\") " pod="openshift-image-registry/node-ca-gfdss" Jul 02 23:01:35.554462 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554385 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-system-cni-dir\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.554462 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554424 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-run-k8s-cni-cncf-io\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.554620 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554463 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-hostroot\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.554620 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554514 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-os-release\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.554620 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554597 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-cni-dir\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.554772 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554642 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8dce277-b4c5-440c-8304-fdcbabbce375-host-slash\") pod \"iptables-alerter-hxllf\" (UID: \"f8dce277-b4c5-440c-8304-fdcbabbce375\") " pod="openshift-network-operator/iptables-alerter-hxllf" Jul 02 23:01:35.554826 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554775 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-var-lib-kubelet\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.554936 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554901 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8dce277-b4c5-440c-8304-fdcbabbce375-iptables-alerter-script\") pod \"iptables-alerter-hxllf\" (UID: \"f8dce277-b4c5-440c-8304-fdcbabbce375\") " pod="openshift-network-operator/iptables-alerter-hxllf" Jul 02 23:01:35.555008 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd-hosts-file\") pod \"node-resolver-4vks2\" (UID: \"12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd\") " pod="openshift-dns/node-resolver-4vks2" Jul 02 23:01:35.555008 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.554979 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-etc-kubernetes\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555078 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555011 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce7afefc-665b-46ab-9895-38facdf8467f-cni-binary-copy\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.555078 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555051 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-var-lib-cni-bin\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555146 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555075 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-daemon-config\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555146 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555076 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ce7afefc-665b-46ab-9895-38facdf8467f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.555146 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555097 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.555146 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555122 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed9789e7-960c-4bb6-b65d-fd51ae63a29e-serviceca\") pod \"node-ca-gfdss\" (UID: \"ed9789e7-960c-4bb6-b65d-fd51ae63a29e\") " pod="openshift-image-registry/node-ca-gfdss" Jul 02 23:01:35.555299 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555150 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-run-multus-certs\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555299 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555175 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ce7afefc-665b-46ab-9895-38facdf8467f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.555299 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555199 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-system-cni-dir\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555299 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555224 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-os-release\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555299 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555257 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f933ee1-d50b-4391-9881-d002c7ef4ab9-cni-binary-copy\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555299 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555282 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-cnibin\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555587 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555314 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-socket-dir-parent\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555587 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-var-lib-cni-multus\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555587 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555382 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-run-netns\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555587 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555404 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd-tmp-dir\") pod \"node-resolver-4vks2\" (UID: \"12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd\") " pod="openshift-dns/node-resolver-4vks2" Jul 02 23:01:35.555587 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555555 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8dce277-b4c5-440c-8304-fdcbabbce375-iptables-alerter-script\") pod \"iptables-alerter-hxllf\" (UID: \"f8dce277-b4c5-440c-8304-fdcbabbce375\") " pod="openshift-network-operator/iptables-alerter-hxllf" Jul 02 23:01:35.555818 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555639 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd-hosts-file\") pod \"node-resolver-4vks2\" (UID: \"12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd\") " pod="openshift-dns/node-resolver-4vks2" Jul 02 23:01:35.555818 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555689 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd-tmp-dir\") pod \"node-resolver-4vks2\" (UID: \"12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd\") " pod="openshift-dns/node-resolver-4vks2" Jul 02 23:01:35.555818 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555756 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-system-cni-dir\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555965 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555818 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-os-release\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.555965 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555855 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-run-multus-certs\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.556056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555982 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-socket-dir-parent\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.556056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.555996 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-etc-kubernetes\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.556056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.556037 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-cnibin\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.556197 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.556060 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ce7afefc-665b-46ab-9895-38facdf8467f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.556197 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.556062 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-var-lib-cni-multus\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.556197 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.556081 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-run-netns\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.556197 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.556097 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f933ee1-d50b-4391-9881-d002c7ef4ab9-host-var-lib-cni-bin\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.556428 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.556245 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce7afefc-665b-46ab-9895-38facdf8467f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.556428 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.556323 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed9789e7-960c-4bb6-b65d-fd51ae63a29e-serviceca\") pod \"node-ca-gfdss\" (UID: \"ed9789e7-960c-4bb6-b65d-fd51ae63a29e\") " pod="openshift-image-registry/node-ca-gfdss" Jul 02 23:01:35.556428 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.556349 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce7afefc-665b-46ab-9895-38facdf8467f-cni-binary-copy\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.556901 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.556858 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9f933ee1-d50b-4391-9881-d002c7ef4ab9-multus-daemon-config\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.556901 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.556892 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f933ee1-d50b-4391-9881-d002c7ef4ab9-cni-binary-copy\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.560721 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.560158 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:01:35.560721 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.560179 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:01:35.560721 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.560193 2568 projected.go:196] Error preparing data for projected volume kube-api-access-kqtqc for pod openshift-network-diagnostics/network-check-target-hzb5r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:35.560721 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.560246 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc podName:cc68e25f-5e72-4518-9f6e-b7b1870bdf97 nodeName:}" failed. No retries permitted until 2026-07-02 23:01:36.060229339 +0000 UTC m=+3.133481703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kqtqc" (UniqueName: "kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc") pod "network-check-target-hzb5r" (UID: "cc68e25f-5e72-4518-9f6e-b7b1870bdf97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:35.562359 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.562327 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9w4j\" (UniqueName: \"kubernetes.io/projected/ed9789e7-960c-4bb6-b65d-fd51ae63a29e-kube-api-access-s9w4j\") pod \"node-ca-gfdss\" (UID: \"ed9789e7-960c-4bb6-b65d-fd51ae63a29e\") " pod="openshift-image-registry/node-ca-gfdss" Jul 02 23:01:35.563371 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.563348 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgxsh\" (UniqueName: \"kubernetes.io/projected/ce7afefc-665b-46ab-9895-38facdf8467f-kube-api-access-wgxsh\") pod \"multus-additional-cni-plugins-xr5tz\" (UID: \"ce7afefc-665b-46ab-9895-38facdf8467f\") " pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.563468 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.563353 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4cv4\" (UniqueName: \"kubernetes.io/projected/f8dce277-b4c5-440c-8304-fdcbabbce375-kube-api-access-h4cv4\") pod \"iptables-alerter-hxllf\" (UID: \"f8dce277-b4c5-440c-8304-fdcbabbce375\") " pod="openshift-network-operator/iptables-alerter-hxllf" Jul 02 23:01:35.563861 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.563841 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n6k5\" (UniqueName: \"kubernetes.io/projected/9f933ee1-d50b-4391-9881-d002c7ef4ab9-kube-api-access-7n6k5\") pod \"multus-k5dh5\" (UID: \"9f933ee1-d50b-4391-9881-d002c7ef4ab9\") " pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.564063 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.564047 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqndp\" (UniqueName: \"kubernetes.io/projected/12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd-kube-api-access-gqndp\") pod \"node-resolver-4vks2\" (UID: \"12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd\") " pod="openshift-dns/node-resolver-4vks2" Jul 02 23:01:35.633616 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.633542 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zf25z" Jul 02 23:01:35.639858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.639816 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:01:35.645024 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:35.645000 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ddcad19_8642_411d_8a79_f815899d94ee.slice/crio-a3ac5642a220ce8af2390acf55f7b9b4fa7ca394bd196cd8907fba623e0ddc5c WatchSource:0}: Error finding container a3ac5642a220ce8af2390acf55f7b9b4fa7ca394bd196cd8907fba623e0ddc5c: Status 404 returned error can't find the container with id a3ac5642a220ce8af2390acf55f7b9b4fa7ca394bd196cd8907fba623e0ddc5c Jul 02 23:01:35.647621 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.647604 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:01:35.649940 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:35.649905 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b1942c_8c3b_4510_bee8_32742e97493a.slice/crio-7161c593d9250908d1f0f28eccc5a88b6ca624f5e598ce43069ab8624e9b27f2 WatchSource:0}: Error finding container 7161c593d9250908d1f0f28eccc5a88b6ca624f5e598ce43069ab8624e9b27f2: Status 404 returned error can't find the container with id 7161c593d9250908d1f0f28eccc5a88b6ca624f5e598ce43069ab8624e9b27f2 Jul 02 23:01:35.655523 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.655488 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" Jul 02 23:01:35.666428 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.666411 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k5dh5" Jul 02 23:01:35.673980 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.673964 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gfdss" Jul 02 23:01:35.682435 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.682375 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hxllf" Jul 02 23:01:35.691545 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.691522 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xr5tz" Jul 02 23:01:35.698074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.698059 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4vks2" Jul 02 23:01:35.760670 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.760649 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:01:35.958290 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.958219 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:35.958446 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:35.958286 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:35.958446 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.958391 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:35.958793 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.958764 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:35.958926 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.958800 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret podName:bb3efb95-e804-4cae-b3b5-220db506cfe8 nodeName:}" failed. No retries permitted until 2026-07-02 23:01:36.958746168 +0000 UTC m=+4.031998512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret") pod "global-pull-secret-syncer-lnsbx" (UID: "bb3efb95-e804-4cae-b3b5-220db506cfe8") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:35.958926 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:35.958849 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs podName:bad930f0-e377-445a-88a5-7e81b222ce9b nodeName:}" failed. No retries permitted until 2026-07-02 23:01:36.958822511 +0000 UTC m=+4.032074866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs") pod "network-metrics-daemon-mm52v" (UID: "bad930f0-e377-445a-88a5-7e81b222ce9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:36.159679 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:36.159645 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqtqc\" (UniqueName: \"kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc\") pod \"network-check-target-hzb5r\" (UID: \"cc68e25f-5e72-4518-9f6e-b7b1870bdf97\") " pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:36.159853 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:36.159799 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:01:36.159853 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:36.159816 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:01:36.159853 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:36.159826 2568 projected.go:196] Error preparing data for projected volume kube-api-access-kqtqc for pod openshift-network-diagnostics/network-check-target-hzb5r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:36.159966 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:36.159885 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc podName:cc68e25f-5e72-4518-9f6e-b7b1870bdf97 nodeName:}" failed. No retries permitted until 2026-07-02 23:01:37.159867596 +0000 UTC m=+4.233119953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqtqc" (UniqueName: "kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc") pod "network-check-target-hzb5r" (UID: "cc68e25f-5e72-4518-9f6e-b7b1870bdf97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:36.389718 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:36.389641 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 22:56:35 +0000 UTC" deadline="2028-03-01 08:58:19.523407438 +0000 UTC" Jul 02 23:01:36.389718 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:36.389714 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14577h56m43.13369785s" Jul 02 23:01:36.462315 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:36.462285 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:36.462446 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:36.462421 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:36.471702 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:36.471667 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" event={"ID":"17b1942c-8c3b-4510-bee8-32742e97493a","Type":"ContainerStarted","Data":"7161c593d9250908d1f0f28eccc5a88b6ca624f5e598ce43069ab8624e9b27f2"} Jul 02 23:01:36.472850 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:36.472828 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zf25z" event={"ID":"8ddcad19-8642-411d-8a79-f815899d94ee","Type":"ContainerStarted","Data":"a3ac5642a220ce8af2390acf55f7b9b4fa7ca394bd196cd8907fba623e0ddc5c"} Jul 02 23:01:36.649310 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:36.649285 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8dce277_b4c5_440c_8304_fdcbabbce375.slice/crio-05c0a9484f56759fcba2293b8ff9dead86823087be1248e204ce598e58bbae6d WatchSource:0}: Error finding container 05c0a9484f56759fcba2293b8ff9dead86823087be1248e204ce598e58bbae6d: Status 404 returned error can't find the container with id 05c0a9484f56759fcba2293b8ff9dead86823087be1248e204ce598e58bbae6d Jul 02 23:01:36.650917 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:36.650801 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f933ee1_d50b_4391_9881_d002c7ef4ab9.slice/crio-82246972ef9023685c36eae8cf8b0407c2a533637eb74edce577a39f251070c4 WatchSource:0}: Error finding container 82246972ef9023685c36eae8cf8b0407c2a533637eb74edce577a39f251070c4: Status 404 returned error can't find the container with id 82246972ef9023685c36eae8cf8b0407c2a533637eb74edce577a39f251070c4 Jul 02 23:01:36.654701 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:36.654680 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dac54d6_06f0_4f56_b1c8_6e24e59ea5af.slice/crio-98acdbcf40fb19abc4ea90215a4a1a86f39d49356d3a50da70bb3570cf288a3d WatchSource:0}: Error finding container 98acdbcf40fb19abc4ea90215a4a1a86f39d49356d3a50da70bb3570cf288a3d: Status 404 returned error can't find the container with id 98acdbcf40fb19abc4ea90215a4a1a86f39d49356d3a50da70bb3570cf288a3d Jul 02 23:01:36.656165 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:36.656127 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded9789e7_960c_4bb6_b65d_fd51ae63a29e.slice/crio-0ba06fd9068fc1f0fa38038ce2e71a05a597f33b9bb7e183ed0ff46429a96f75 WatchSource:0}: Error finding container 0ba06fd9068fc1f0fa38038ce2e71a05a597f33b9bb7e183ed0ff46429a96f75: Status 404 returned error can't find the container with id 0ba06fd9068fc1f0fa38038ce2e71a05a597f33b9bb7e183ed0ff46429a96f75 Jul 02 23:01:36.657095 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:36.657068 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23cd16b1_4932_4ef3_b5ad_e1d4a48650d7.slice/crio-248fe664b316789db46808d9b1e8470e9eb294516f92f08300b35573f4b09a3e WatchSource:0}: Error finding container 248fe664b316789db46808d9b1e8470e9eb294516f92f08300b35573f4b09a3e: Status 404 returned error can't find the container with id 248fe664b316789db46808d9b1e8470e9eb294516f92f08300b35573f4b09a3e Jul 02 23:01:36.659451 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:36.659315 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a6f254_8a6d_4f61_a3c9_220fb8d9f1fd.slice/crio-05667ac858391e767d4939b58d1d8bec4157942f59d6127902287f4fb476d3f1 WatchSource:0}: Error finding container 05667ac858391e767d4939b58d1d8bec4157942f59d6127902287f4fb476d3f1: Status 404 returned error can't find the container with id 05667ac858391e767d4939b58d1d8bec4157942f59d6127902287f4fb476d3f1 Jul 02 23:01:36.660836 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:01:36.660817 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce7afefc_665b_46ab_9895_38facdf8467f.slice/crio-2e8c2ab6a4897d448ea277f73505bb990a30f207dba080e91f70a90b6e4f5d75 WatchSource:0}: Error finding container 2e8c2ab6a4897d448ea277f73505bb990a30f207dba080e91f70a90b6e4f5d75: Status 404 returned error can't find the container with id 2e8c2ab6a4897d448ea277f73505bb990a30f207dba080e91f70a90b6e4f5d75 Jul 02 23:01:36.965053 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:36.964771 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:36.965053 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:36.965045 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:36.965227 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:36.964876 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:36.965227 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:36.965136 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs podName:bad930f0-e377-445a-88a5-7e81b222ce9b nodeName:}" failed. No retries permitted until 2026-07-02 23:01:38.965115186 +0000 UTC m=+6.038367543 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs") pod "network-metrics-daemon-mm52v" (UID: "bad930f0-e377-445a-88a5-7e81b222ce9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:36.965227 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:36.965169 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:36.965227 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:36.965204 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret podName:bb3efb95-e804-4cae-b3b5-220db506cfe8 nodeName:}" failed. No retries permitted until 2026-07-02 23:01:38.965193698 +0000 UTC m=+6.038446054 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret") pod "global-pull-secret-syncer-lnsbx" (UID: "bb3efb95-e804-4cae-b3b5-220db506cfe8") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:37.166650 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.166615 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqtqc\" (UniqueName: \"kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc\") pod \"network-check-target-hzb5r\" (UID: \"cc68e25f-5e72-4518-9f6e-b7b1870bdf97\") " pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:37.166798 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:37.166783 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:01:37.166855 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:37.166801 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:01:37.166855 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:37.166814 2568 projected.go:196] Error preparing data for projected volume kube-api-access-kqtqc for pod openshift-network-diagnostics/network-check-target-hzb5r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:37.166961 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:37.166867 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc podName:cc68e25f-5e72-4518-9f6e-b7b1870bdf97 nodeName:}" failed. No retries permitted until 2026-07-02 23:01:39.166849619 +0000 UTC m=+6.240101963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqtqc" (UniqueName: "kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc") pod "network-check-target-hzb5r" (UID: "cc68e25f-5e72-4518-9f6e-b7b1870bdf97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:37.390458 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.390418 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 22:56:35 +0000 UTC" deadline="2028-04-17 00:43:31.597581352 +0000 UTC" Jul 02 23:01:37.390458 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.390458 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15697h41m54.207128713s" Jul 02 23:01:37.464699 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.464675 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:37.464699 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.464689 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:37.464895 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:37.464796 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:37.464957 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:37.464932 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:37.480292 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.480234 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4vks2" event={"ID":"12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd","Type":"ContainerStarted","Data":"05667ac858391e767d4939b58d1d8bec4157942f59d6127902287f4fb476d3f1"} Jul 02 23:01:37.481855 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.481787 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-77nbd" event={"ID":"1dac54d6-06f0-4f56-b1c8-6e24e59ea5af","Type":"ContainerStarted","Data":"98acdbcf40fb19abc4ea90215a4a1a86f39d49356d3a50da70bb3570cf288a3d"} Jul 02 23:01:37.488068 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.488038 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k5dh5" event={"ID":"9f933ee1-d50b-4391-9881-d002c7ef4ab9","Type":"ContainerStarted","Data":"82246972ef9023685c36eae8cf8b0407c2a533637eb74edce577a39f251070c4"} Jul 02 23:01:37.496898 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.496876 2568 generic.go:356] "Generic (PLEG): container finished" podID="168cb0651600516a255a42a3464e8a1f" containerID="5155accfb34a955e12c9053b4484e143e8e10b1f82cfe4b49eb9e3e9eb262f54" exitCode=0 Jul 02 23:01:37.497007 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.496944 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" event={"ID":"168cb0651600516a255a42a3464e8a1f","Type":"ContainerDied","Data":"5155accfb34a955e12c9053b4484e143e8e10b1f82cfe4b49eb9e3e9eb262f54"} Jul 02 23:01:37.507503 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.507453 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gfdss" event={"ID":"ed9789e7-960c-4bb6-b65d-fd51ae63a29e","Type":"ContainerStarted","Data":"0ba06fd9068fc1f0fa38038ce2e71a05a597f33b9bb7e183ed0ff46429a96f75"} Jul 02 23:01:37.512263 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.512215 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hxllf" event={"ID":"f8dce277-b4c5-440c-8304-fdcbabbce375","Type":"ContainerStarted","Data":"05c0a9484f56759fcba2293b8ff9dead86823087be1248e204ce598e58bbae6d"} Jul 02 23:01:37.517555 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.517529 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-71.ec2.internal" event={"ID":"934cc245ae5a1b254602b20dfff22029","Type":"ContainerStarted","Data":"e045c16e00ac713fc3952eea95b4ad677eada859dcc489c99c606a563225ff5f"} Jul 02 23:01:37.528637 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.528559 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xr5tz" event={"ID":"ce7afefc-665b-46ab-9895-38facdf8467f","Type":"ContainerStarted","Data":"2e8c2ab6a4897d448ea277f73505bb990a30f207dba080e91f70a90b6e4f5d75"} Jul 02 23:01:37.532246 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.532060 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-71.ec2.internal" podStartSLOduration=3.532042407 podStartE2EDuration="3.532042407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:01:37.530891077 +0000 UTC m=+4.604143441" watchObservedRunningTime="2026-07-02 23:01:37.532042407 +0000 UTC m=+4.605294772" Jul 02 23:01:37.536448 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:37.536102 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" event={"ID":"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7","Type":"ContainerStarted","Data":"248fe664b316789db46808d9b1e8470e9eb294516f92f08300b35573f4b09a3e"} Jul 02 23:01:38.463028 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:38.462986 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:38.463465 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:38.463126 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:38.563598 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:38.563433 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" event={"ID":"168cb0651600516a255a42a3464e8a1f","Type":"ContainerStarted","Data":"4fb0d88679b7298e8b653eb97b85780b04cb7905b4780a39478e700736f246ee"} Jul 02 23:01:38.982130 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:38.981350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:38.982130 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:38.981434 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:38.982130 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:38.981550 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:38.982130 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:38.981625 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs podName:bad930f0-e377-445a-88a5-7e81b222ce9b nodeName:}" failed. No retries permitted until 2026-07-02 23:01:42.981605799 +0000 UTC m=+10.054858143 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs") pod "network-metrics-daemon-mm52v" (UID: "bad930f0-e377-445a-88a5-7e81b222ce9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:38.982130 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:38.982006 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:38.982130 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:38.982055 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret podName:bb3efb95-e804-4cae-b3b5-220db506cfe8 nodeName:}" failed. No retries permitted until 2026-07-02 23:01:42.982038775 +0000 UTC m=+10.055291128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret") pod "global-pull-secret-syncer-lnsbx" (UID: "bb3efb95-e804-4cae-b3b5-220db506cfe8") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:39.182592 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:39.182521 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqtqc\" (UniqueName: \"kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc\") pod \"network-check-target-hzb5r\" (UID: \"cc68e25f-5e72-4518-9f6e-b7b1870bdf97\") " pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:39.182766 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:39.182717 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:01:39.182766 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:39.182738 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:01:39.182766 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:39.182751 2568 projected.go:196] Error preparing data for projected volume kube-api-access-kqtqc for pod openshift-network-diagnostics/network-check-target-hzb5r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:39.182915 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:39.182807 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc podName:cc68e25f-5e72-4518-9f6e-b7b1870bdf97 nodeName:}" failed. No retries permitted until 2026-07-02 23:01:43.182788029 +0000 UTC m=+10.256040371 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqtqc" (UniqueName: "kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc") pod "network-check-target-hzb5r" (UID: "cc68e25f-5e72-4518-9f6e-b7b1870bdf97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:39.463686 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:39.463655 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:39.464136 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:39.463785 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:39.464209 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:39.464159 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:39.464279 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:39.464251 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:40.462774 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:40.462732 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:40.462910 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:40.462890 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:41.463431 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:41.463399 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:41.463892 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:41.463532 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:41.463959 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:41.463920 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:41.464034 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:41.464006 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:42.462800 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:42.462096 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:42.462800 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:42.462233 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:43.014908 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:43.014860 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:43.015370 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:43.014925 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:43.015370 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:43.015002 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:43.015370 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:43.015053 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:43.015370 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:43.015077 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs podName:bad930f0-e377-445a-88a5-7e81b222ce9b nodeName:}" failed. No retries permitted until 2026-07-02 23:01:51.015054685 +0000 UTC m=+18.088307037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs") pod "network-metrics-daemon-mm52v" (UID: "bad930f0-e377-445a-88a5-7e81b222ce9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:43.015370 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:43.015192 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret podName:bb3efb95-e804-4cae-b3b5-220db506cfe8 nodeName:}" failed. No retries permitted until 2026-07-02 23:01:51.015168897 +0000 UTC m=+18.088421252 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret") pod "global-pull-secret-syncer-lnsbx" (UID: "bb3efb95-e804-4cae-b3b5-220db506cfe8") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:43.216755 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:43.216436 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqtqc\" (UniqueName: \"kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc\") pod \"network-check-target-hzb5r\" (UID: \"cc68e25f-5e72-4518-9f6e-b7b1870bdf97\") " pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:43.216755 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:43.216626 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:01:43.216755 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:43.216647 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:01:43.216755 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:43.216660 2568 projected.go:196] Error preparing data for projected volume kube-api-access-kqtqc for pod openshift-network-diagnostics/network-check-target-hzb5r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:43.216755 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:43.216717 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc podName:cc68e25f-5e72-4518-9f6e-b7b1870bdf97 nodeName:}" failed. No retries permitted until 2026-07-02 23:01:51.216698954 +0000 UTC m=+18.289951308 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqtqc" (UniqueName: "kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc") pod "network-check-target-hzb5r" (UID: "cc68e25f-5e72-4518-9f6e-b7b1870bdf97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:43.464052 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:43.462734 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:43.464052 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:43.462820 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:43.464052 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:43.463175 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:43.464052 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:43.463267 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:44.462652 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:44.462614 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:44.463067 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:44.462755 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:45.462140 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:45.462102 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:45.462797 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:45.462324 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:45.462797 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:45.462370 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:45.462797 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:45.462445 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:46.462874 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:46.462836 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:46.463259 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:46.462984 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:47.462624 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:47.462593 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:47.462799 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:47.462727 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:47.462799 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:47.462787 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:47.462931 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:47.462911 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:48.462036 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:48.462004 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:48.462176 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:48.462140 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:49.462722 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:49.462686 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:49.463173 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:49.462813 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:49.463173 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:49.463038 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:49.463173 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:49.463142 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:50.462105 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:50.462077 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:50.462291 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:50.462189 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:51.074387 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:51.074346 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:51.074844 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:51.074397 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:51.074844 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:51.074464 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:51.074844 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:51.074480 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:51.074844 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:51.074535 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs podName:bad930f0-e377-445a-88a5-7e81b222ce9b nodeName:}" failed. No retries permitted until 2026-07-02 23:02:07.074514555 +0000 UTC m=+34.147766900 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs") pod "network-metrics-daemon-mm52v" (UID: "bad930f0-e377-445a-88a5-7e81b222ce9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:01:51.074844 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:51.074555 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret podName:bb3efb95-e804-4cae-b3b5-220db506cfe8 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:07.074545989 +0000 UTC m=+34.147798333 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret") pod "global-pull-secret-syncer-lnsbx" (UID: "bb3efb95-e804-4cae-b3b5-220db506cfe8") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:01:51.275439 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:51.275408 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqtqc\" (UniqueName: \"kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc\") pod \"network-check-target-hzb5r\" (UID: \"cc68e25f-5e72-4518-9f6e-b7b1870bdf97\") " pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:51.275610 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:51.275598 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:01:51.275662 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:51.275617 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:01:51.275662 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:51.275627 2568 projected.go:196] Error preparing data for projected volume kube-api-access-kqtqc for pod openshift-network-diagnostics/network-check-target-hzb5r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:51.275731 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:51.275688 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc podName:cc68e25f-5e72-4518-9f6e-b7b1870bdf97 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:07.275670442 +0000 UTC m=+34.348922800 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqtqc" (UniqueName: "kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc") pod "network-check-target-hzb5r" (UID: "cc68e25f-5e72-4518-9f6e-b7b1870bdf97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:01:51.462467 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:51.462435 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:51.462659 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:51.462582 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:51.462659 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:51.462625 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:51.462777 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:51.462736 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:52.462878 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:52.462846 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:52.463338 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:52.462966 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:53.463744 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:53.463717 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:53.464272 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:53.463794 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:53.464406 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:53.464274 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:53.464406 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:53.464331 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:54.462704 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:54.462673 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:54.462867 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:54.462776 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:55.462319 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.462195 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:55.462944 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.462208 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:55.462944 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:55.462417 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:55.462944 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:55.462519 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:55.596782 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.596758 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-77nbd" event={"ID":"1dac54d6-06f0-4f56-b1c8-6e24e59ea5af","Type":"ContainerStarted","Data":"0d07e91f159c83ea5f869b3c48cfb30cf844a88868c5142daaaa99c69bd7ecd3"} Jul 02 23:01:55.598137 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.598117 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k5dh5" event={"ID":"9f933ee1-d50b-4391-9881-d002c7ef4ab9","Type":"ContainerStarted","Data":"b216c111796b6bd23ad39de822fee89353885c61096dea664b6ec7f9f108983a"} Jul 02 23:01:55.599431 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.599399 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gfdss" event={"ID":"ed9789e7-960c-4bb6-b65d-fd51ae63a29e","Type":"ContainerStarted","Data":"77b8404e228a55d26e2d5dc3781683740c9585342b9cbf912af2a7860ec7b1e3"} Jul 02 23:01:55.601663 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.601645 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:01:55.601936 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.601917 2568 generic.go:356] "Generic (PLEG): container finished" podID="17b1942c-8c3b-4510-bee8-32742e97493a" containerID="a0553fa60eb9681b1a43599146ae566a5827e17e96b65d2980975d8227e1c5a2" exitCode=1 Jul 02 23:01:55.602001 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.601977 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" event={"ID":"17b1942c-8c3b-4510-bee8-32742e97493a","Type":"ContainerStarted","Data":"9c48fad848961f166c4a2bc6c5b90dc8d5b1dbdf716eb832561dde22ee2a2800"} Jul 02 23:01:55.602054 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.602039 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" event={"ID":"17b1942c-8c3b-4510-bee8-32742e97493a","Type":"ContainerStarted","Data":"94701ebc11db11da7da9fc9b608a9cc7cc56750db4a422e15e1c328cb1f00b33"} Jul 02 23:01:55.602106 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.602094 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" event={"ID":"17b1942c-8c3b-4510-bee8-32742e97493a","Type":"ContainerStarted","Data":"fb8dbc9aa9d0bb45e1ac1aecda2e62f098b4f92cb2695bcac2982e83083be1ed"} Jul 02 23:01:55.602158 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.602145 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" event={"ID":"17b1942c-8c3b-4510-bee8-32742e97493a","Type":"ContainerDied","Data":"a0553fa60eb9681b1a43599146ae566a5827e17e96b65d2980975d8227e1c5a2"} Jul 02 23:01:55.602199 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.602187 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" event={"ID":"17b1942c-8c3b-4510-bee8-32742e97493a","Type":"ContainerStarted","Data":"7c2f23c3844c5d3c43468583abd7ac505836dcdf13456c4ab671ed746cdc1d90"} Jul 02 23:01:55.603332 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.603312 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zf25z" event={"ID":"8ddcad19-8642-411d-8a79-f815899d94ee","Type":"ContainerStarted","Data":"90442204ee1b9de9763416e5d229220b3924a125fd2564e97f2064fcf9d80251"} Jul 02 23:01:55.608119 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.608096 2568 generic.go:356] "Generic (PLEG): container finished" podID="ce7afefc-665b-46ab-9895-38facdf8467f" containerID="be3ecb857f797936bd49fcb767422a5eb36e16dbbbe16b93cec2cca3e6449de2" exitCode=0 Jul 02 23:01:55.608201 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.608130 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xr5tz" event={"ID":"ce7afefc-665b-46ab-9895-38facdf8467f","Type":"ContainerDied","Data":"be3ecb857f797936bd49fcb767422a5eb36e16dbbbe16b93cec2cca3e6449de2"} Jul 02 23:01:55.609602 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.609547 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" event={"ID":"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7","Type":"ContainerStarted","Data":"30e07aa3aee11808cacb9d117140a81aba08014c87dc64b8b36c4e813357126a"} Jul 02 23:01:55.610878 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.610858 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4vks2" event={"ID":"12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd","Type":"ContainerStarted","Data":"4ff6e086ad5a5127a6a777a9b56653af378164a27e0cb3df46274e5c7bb75f29"} Jul 02 23:01:55.620466 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.620433 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-77nbd" podStartSLOduration=4.502472126 podStartE2EDuration="22.620422534s" totalImagesPullingTime="18.117950408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:01:55.620216903 +0000 UTC m=+22.693469265" watchObservedRunningTime="2026-07-02 23:01:55.620422534 +0000 UTC m=+22.693674924" Jul 02 23:01:55.620608 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.620589 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-71.ec2.internal" podStartSLOduration=21.620584567999998 podStartE2EDuration="21.620584568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:01:38.579711382 +0000 UTC m=+5.652963748" watchObservedRunningTime="2026-07-02 23:01:55.620584568 +0000 UTC m=+22.693836922" Jul 02 23:01:55.637239 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.637200 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gfdss" podStartSLOduration=4.521458799 podStartE2EDuration="22.637185584s" totalImagesPullingTime="18.115726785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:01:55.637158071 +0000 UTC m=+22.710410434" watchObservedRunningTime="2026-07-02 23:01:55.637185584 +0000 UTC m=+22.710437952" Jul 02 23:01:55.683739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.683700 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k5dh5" podStartSLOduration=4.543708496 podStartE2EDuration="22.683688747s" totalImagesPullingTime="18.139980251s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:01:55.683393819 +0000 UTC m=+22.756646182" watchObservedRunningTime="2026-07-02 23:01:55.683688747 +0000 UTC m=+22.756941110" Jul 02 23:01:55.699148 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.699110 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4vks2" podStartSLOduration=4.586461095 podStartE2EDuration="22.69909854s" totalImagesPullingTime="18.112637445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:01:55.698927154 +0000 UTC m=+22.772179516" watchObservedRunningTime="2026-07-02 23:01:55.69909854 +0000 UTC m=+22.772350900" Jul 02 23:01:55.721226 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.721190 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zf25z" podStartSLOduration=3.592678742 podStartE2EDuration="22.721179821s" totalImagesPullingTime="19.128501079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:01:55.72068442 +0000 UTC m=+22.793936783" watchObservedRunningTime="2026-07-02 23:01:55.721179821 +0000 UTC m=+22.794432183" Jul 02 23:01:55.844015 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.843950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:55.926218 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:55.926185 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:56.354655 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.354624 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 23:01:56.433932 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.433857 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T23:01:56.354640832Z","UUID":"a81e853b-e16e-489b-8e34-40e8fe103b61","Handler":null,"Name":"","Endpoint":""} Jul 02 23:01:56.435515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.435494 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 23:01:56.435686 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.435535 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 23:01:56.461989 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.461968 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:56.462108 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:56.462083 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:56.615452 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.615429 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hxllf" event={"ID":"f8dce277-b4c5-440c-8304-fdcbabbce375","Type":"ContainerStarted","Data":"3c10aa0bacf120e53ed049db86e4b5d0a7bcc2f60964035b1e176b2571a90413"} Jul 02 23:01:56.618812 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.618784 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:01:56.619251 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.619219 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" event={"ID":"17b1942c-8c3b-4510-bee8-32742e97493a","Type":"ContainerStarted","Data":"46335be5488affeddc61c40b2c8f4857604367dac6e2496694a40d3ba6698776"} Jul 02 23:01:56.621137 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.621115 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" event={"ID":"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7","Type":"ContainerStarted","Data":"f18e8b37bee9c81b77cf5d4a8b365a119b417032283b4a25aae2e97e89353a77"} Jul 02 23:01:56.632087 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.632039 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hxllf" podStartSLOduration=5.508822636 podStartE2EDuration="23.632022233s" totalImagesPullingTime="18.123199597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:01:56.631343465 +0000 UTC m=+23.704595828" watchObservedRunningTime="2026-07-02 23:01:56.632022233 +0000 UTC m=+23.705274599" Jul 02 23:01:56.751456 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.751426 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:56.814206 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.814174 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:56.950679 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:56.950589 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:57.006380 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:57.006349 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:57.172494 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:57.172464 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:57.237325 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:57.237257 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:57.463591 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:57.463397 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:57.463706 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:57.463686 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:57.463763 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:57.463733 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:57.463809 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:57.463796 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:57.625793 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:57.625764 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" event={"ID":"23cd16b1-4932-4ef3-b5ad-e1d4a48650d7","Type":"ContainerStarted","Data":"3a5cb3b69010b61bc7b3f2a97af4fd65aafbfe15b180eed6a0e1484ede072763"} Jul 02 23:01:57.646132 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:57.646087 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwgmf" podStartSLOduration=4.287392807 podStartE2EDuration="24.646071283s" totalImagesPullingTime="20.358678476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:01:57.645280473 +0000 UTC m=+24.718532837" watchObservedRunningTime="2026-07-02 23:01:57.646071283 +0000 UTC m=+24.719323647" Jul 02 23:01:57.951723 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:57.951647 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:58.049559 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:58.049524 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:58.355009 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:58.354932 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:58.408893 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:58.408863 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:58.462990 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:58.462968 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:01:58.463100 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:58.463063 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:01:58.631865 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:58.631837 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:01:58.632458 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:58.632245 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" event={"ID":"17b1942c-8c3b-4510-bee8-32742e97493a","Type":"ContainerStarted","Data":"c860ba78264ef069b5f336dac9ba3278fa322d1c1ab7a6f0e3e15aebe06ad2dd"} Jul 02 23:01:59.383841 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:59.383810 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:01:59.462325 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:59.462299 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:01:59.462325 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:59.462314 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:01:59.462552 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:59.462427 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:01:59.462639 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:01:59.462556 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:01:59.484402 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:01:59.484365 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:00.158008 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:00.157978 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:02:00.158694 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:00.158672 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:02:00.462116 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:00.462049 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:02:00.462255 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:00.462154 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:02:00.842414 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:00.842389 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:01.462227 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.462036 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:02:01.462587 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:01.462276 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:02:01.462683 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.462663 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:02:01.462901 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:01.462758 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:02:01.642224 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.642200 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:02:01.642623 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.642603 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" event={"ID":"17b1942c-8c3b-4510-bee8-32742e97493a","Type":"ContainerStarted","Data":"9dd91f9b69b5b65e62b4bc019d8f28c540cf8412c90c66001a2336019a73b383"} Jul 02 23:02:01.642908 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.642887 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:02:01.643024 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.642963 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:02:01.643089 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.643052 2568 scope.go:117] "RemoveContainer" containerID="a0553fa60eb9681b1a43599146ae566a5827e17e96b65d2980975d8227e1c5a2" Jul 02 23:02:01.644804 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.644775 2568 generic.go:356] "Generic (PLEG): container finished" podID="ce7afefc-665b-46ab-9895-38facdf8467f" containerID="0c2300f2a3f2e112d7b8a0ac596d65f8f391ef1a8940bd61a64bacf2cb5a3b06" exitCode=0 Jul 02 23:02:01.644891 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.644827 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xr5tz" event={"ID":"ce7afefc-665b-46ab-9895-38facdf8467f","Type":"ContainerDied","Data":"0c2300f2a3f2e112d7b8a0ac596d65f8f391ef1a8940bd61a64bacf2cb5a3b06"} Jul 02 23:02:01.658963 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.658937 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:02:01.811459 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.811434 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:01.879631 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:01.879608 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:02.380121 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.380094 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:02.436973 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.436943 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:02.462478 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.462453 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:02:02.462868 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:02.462559 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:02:02.622734 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.622704 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:02:02.622921 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.622843 2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 02 23:02:02.623349 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.623322 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-77nbd" Jul 02 23:02:02.632976 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.632928 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:02.650712 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.650693 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:02:02.651032 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.651009 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" event={"ID":"17b1942c-8c3b-4510-bee8-32742e97493a","Type":"ContainerStarted","Data":"62e0a9eda9df549f78dd3e20d15d893f67e2a9a7e7c59240b6f26da54f2138cc"} Jul 02 23:02:02.651321 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.651299 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:02:02.665473 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.665455 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:02:02.677678 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.677620 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" podStartSLOduration=10.236835453 podStartE2EDuration="29.67760352s" totalImagesPullingTime="19.440768067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:02.675560699 +0000 UTC m=+29.748813063" watchObservedRunningTime="2026-07-02 23:02:02.67760352 +0000 UTC m=+29.750855881" Jul 02 23:02:02.705001 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:02.704977 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:03.034250 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:03.034079 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hzb5r"] Jul 02 23:02:03.034403 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:03.034354 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:02:03.034483 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:03.034459 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:02:03.034848 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:03.034832 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lnsbx"] Jul 02 23:02:03.034918 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:03.034908 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:02:03.034992 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:03.034977 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:02:03.047104 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:03.047082 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mm52v"] Jul 02 23:02:03.047223 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:03.047160 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:02:03.047284 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:03.047261 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:02:03.654586 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:03.654534 2568 generic.go:356] "Generic (PLEG): container finished" podID="ce7afefc-665b-46ab-9895-38facdf8467f" containerID="fb61e2989ff9604ff24a1f824bccb832da5bd169699b082c7bee65a40cf2ba90" exitCode=0 Jul 02 23:02:03.655106 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:03.654604 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xr5tz" event={"ID":"ce7afefc-665b-46ab-9895-38facdf8467f","Type":"ContainerDied","Data":"fb61e2989ff9604ff24a1f824bccb832da5bd169699b082c7bee65a40cf2ba90"} Jul 02 23:02:04.462637 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:04.462607 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:02:04.462787 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:04.462618 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:02:04.462787 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:04.462623 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:02:04.462787 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:04.462727 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:02:04.462925 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:04.462806 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:02:04.462925 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:04.462882 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:02:05.661367 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:05.661333 2568 generic.go:356] "Generic (PLEG): container finished" podID="ce7afefc-665b-46ab-9895-38facdf8467f" containerID="444e71f0fb697bc9c0cd683aa4ad40a893868e5acd59a99e48c071640a242d33" exitCode=0 Jul 02 23:02:05.661893 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:05.661391 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xr5tz" event={"ID":"ce7afefc-665b-46ab-9895-38facdf8467f","Type":"ContainerDied","Data":"444e71f0fb697bc9c0cd683aa4ad40a893868e5acd59a99e48c071640a242d33"} Jul 02 23:02:06.462214 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:06.462182 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:02:06.462380 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:06.462186 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:02:06.462380 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:06.462289 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lnsbx" podUID="bb3efb95-e804-4cae-b3b5-220db506cfe8" Jul 02 23:02:06.462380 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:06.462186 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:02:06.462536 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:06.462375 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzb5r" podUID="cc68e25f-5e72-4518-9f6e-b7b1870bdf97" Jul 02 23:02:06.462536 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:06.462463 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mm52v" podUID="bad930f0-e377-445a-88a5-7e81b222ce9b" Jul 02 23:02:07.103547 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.103512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:02:07.103985 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.103557 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:02:07.103985 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:07.103686 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:02:07.103985 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:07.103744 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs podName:bad930f0-e377-445a-88a5-7e81b222ce9b nodeName:}" failed. No retries permitted until 2026-07-02 23:02:39.103727273 +0000 UTC m=+66.176979614 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs") pod "network-metrics-daemon-mm52v" (UID: "bad930f0-e377-445a-88a5-7e81b222ce9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:02:07.103985 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:07.103801 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:02:07.103985 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:07.103866 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret podName:bb3efb95-e804-4cae-b3b5-220db506cfe8 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:39.103849556 +0000 UTC m=+66.177101899 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret") pod "global-pull-secret-syncer-lnsbx" (UID: "bb3efb95-e804-4cae-b3b5-220db506cfe8") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:02:07.305025 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.304996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqtqc\" (UniqueName: \"kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc\") pod \"network-check-target-hzb5r\" (UID: \"cc68e25f-5e72-4518-9f6e-b7b1870bdf97\") " pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:02:07.305186 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:07.305150 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:02:07.305186 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:07.305169 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:02:07.305186 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:07.305182 2568 projected.go:196] Error preparing data for projected volume kube-api-access-kqtqc for pod openshift-network-diagnostics/network-check-target-hzb5r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:02:07.305309 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:07.305237 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc podName:cc68e25f-5e72-4518-9f6e-b7b1870bdf97 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:39.305222749 +0000 UTC m=+66.378475090 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqtqc" (UniqueName: "kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc") pod "network-check-target-hzb5r" (UID: "cc68e25f-5e72-4518-9f6e-b7b1870bdf97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:02:07.691610 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.691523 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-71.ec2.internal" event="NodeReady" Jul 02 23:02:07.691765 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.691669 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 23:02:07.729318 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.729290 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84f6dfc7cc-pjfxq"] Jul 02 23:02:07.731187 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.731162 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:07.733666 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.733642 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 23:02:07.734237 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.734205 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 23:02:07.734477 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.734458 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mj5r6\"" Jul 02 23:02:07.734632 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.734515 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 23:02:07.738389 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.738324 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-d6kx2"] Jul 02 23:02:07.740174 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.740008 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:07.740971 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.740953 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 23:02:07.741184 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.741165 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kdqh7"] Jul 02 23:02:07.742505 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.742487 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6szd7\"" Jul 02 23:02:07.742740 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.742719 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 23:02:07.742838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.742769 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 23:02:07.743607 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.743589 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kdqh7" Jul 02 23:02:07.746541 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.746071 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qbct9\"" Jul 02 23:02:07.746541 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.746235 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 23:02:07.746541 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.746460 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 23:02:07.746838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.746814 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 23:02:07.747462 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.747286 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84f6dfc7cc-pjfxq"] Jul 02 23:02:07.751188 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.751136 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d6kx2"] Jul 02 23:02:07.753389 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.753372 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kdqh7"] Jul 02 23:02:07.865447 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.865398 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:07.908382 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908351 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-image-registry-private-configuration\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:07.908382 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908385 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:07.908597 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908411 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54bffe79-6186-4756-a955-f21219245e06-cert\") pod \"ingress-canary-kdqh7\" (UID: \"54bffe79-6186-4756-a955-f21219245e06\") " pod="openshift-ingress-canary/ingress-canary-kdqh7" Jul 02 23:02:07.908597 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908459 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:07.908597 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908542 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72042bb7-03a7-4133-b397-ecb5c0052cea-ca-trust-extracted\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:07.908745 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908607 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbzdl\" (UniqueName: \"kubernetes.io/projected/759fdb13-b1f4-4476-8a9d-2b52e33105aa-kube-api-access-xbzdl\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:07.908745 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908638 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm4h2\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-kube-api-access-jm4h2\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:07.908745 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908702 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/759fdb13-b1f4-4476-8a9d-2b52e33105aa-tmp-dir\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:07.908871 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908807 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlnhc\" (UniqueName: \"kubernetes.io/projected/54bffe79-6186-4756-a955-f21219245e06-kube-api-access-vlnhc\") pod \"ingress-canary-kdqh7\" (UID: \"54bffe79-6186-4756-a955-f21219245e06\") " pod="openshift-ingress-canary/ingress-canary-kdqh7" Jul 02 23:02:07.908871 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908831 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-trusted-ca\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:07.908871 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908856 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-installation-pull-secrets\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:07.908984 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908884 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/759fdb13-b1f4-4476-8a9d-2b52e33105aa-config-volume\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:07.908984 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908907 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-certificates\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:07.908984 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:07.908944 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-bound-sa-token\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.009778 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.009706 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbzdl\" (UniqueName: \"kubernetes.io/projected/759fdb13-b1f4-4476-8a9d-2b52e33105aa-kube-api-access-xbzdl\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:08.009918 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.009805 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm4h2\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-kube-api-access-jm4h2\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.009918 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.009844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/759fdb13-b1f4-4476-8a9d-2b52e33105aa-tmp-dir\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:08.009918 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.009864 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlnhc\" (UniqueName: \"kubernetes.io/projected/54bffe79-6186-4756-a955-f21219245e06-kube-api-access-vlnhc\") pod \"ingress-canary-kdqh7\" (UID: \"54bffe79-6186-4756-a955-f21219245e06\") " pod="openshift-ingress-canary/ingress-canary-kdqh7" Jul 02 23:02:08.009918 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.009898 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-trusted-ca\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.010121 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.009921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-installation-pull-secrets\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.010121 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.010031 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/759fdb13-b1f4-4476-8a9d-2b52e33105aa-config-volume\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:08.010121 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.010075 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-certificates\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.010261 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.010130 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-bound-sa-token\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.010261 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.010220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-image-registry-private-configuration\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.010261 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.010252 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:08.010410 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.010264 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/759fdb13-b1f4-4476-8a9d-2b52e33105aa-tmp-dir\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:08.010410 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.010277 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54bffe79-6186-4756-a955-f21219245e06-cert\") pod \"ingress-canary-kdqh7\" (UID: \"54bffe79-6186-4756-a955-f21219245e06\") " pod="openshift-ingress-canary/ingress-canary-kdqh7" Jul 02 23:02:08.010410 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.010302 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.010410 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.010344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72042bb7-03a7-4133-b397-ecb5c0052cea-ca-trust-extracted\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.011020 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:08.010642 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:02:08.011020 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:08.010669 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f6dfc7cc-pjfxq: secret "image-registry-tls" not found Jul 02 23:02:08.011020 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.010692 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72042bb7-03a7-4133-b397-ecb5c0052cea-ca-trust-extracted\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.011020 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:08.010738 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:02:08.011020 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:08.010755 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls podName:72042bb7-03a7-4133-b397-ecb5c0052cea nodeName:}" failed. No retries permitted until 2026-07-02 23:02:08.510730943 +0000 UTC m=+35.583983285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls") pod "image-registry-84f6dfc7cc-pjfxq" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea") : secret "image-registry-tls" not found Jul 02 23:02:08.011020 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:08.010820 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls podName:759fdb13-b1f4-4476-8a9d-2b52e33105aa nodeName:}" failed. No retries permitted until 2026-07-02 23:02:08.510798158 +0000 UTC m=+35.584050507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls") pod "dns-default-d6kx2" (UID: "759fdb13-b1f4-4476-8a9d-2b52e33105aa") : secret "dns-default-metrics-tls" not found Jul 02 23:02:08.011359 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.011027 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-trusted-ca\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.011359 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.011217 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-certificates\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.011448 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.011359 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/759fdb13-b1f4-4476-8a9d-2b52e33105aa-config-volume\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:08.014611 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.014588 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-installation-pull-secrets\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.014711 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.014620 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-image-registry-private-configuration\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.014711 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.014621 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54bffe79-6186-4756-a955-f21219245e06-cert\") pod \"ingress-canary-kdqh7\" (UID: \"54bffe79-6186-4756-a955-f21219245e06\") " pod="openshift-ingress-canary/ingress-canary-kdqh7" Jul 02 23:02:08.018147 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.018130 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlnhc\" (UniqueName: \"kubernetes.io/projected/54bffe79-6186-4756-a955-f21219245e06-kube-api-access-vlnhc\") pod \"ingress-canary-kdqh7\" (UID: \"54bffe79-6186-4756-a955-f21219245e06\") " pod="openshift-ingress-canary/ingress-canary-kdqh7" Jul 02 23:02:08.021578 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.021540 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm4h2\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-kube-api-access-jm4h2\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.021578 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.021556 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-bound-sa-token\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.023002 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.022966 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbzdl\" (UniqueName: \"kubernetes.io/projected/759fdb13-b1f4-4476-8a9d-2b52e33105aa-kube-api-access-xbzdl\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:08.060619 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.060590 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kdqh7" Jul 02 23:02:08.163457 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.163262 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:08.201231 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.201206 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kdqh7"] Jul 02 23:02:08.205674 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:08.205634 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54bffe79_6186_4756_a955_f21219245e06.slice/crio-9fd733220ff3e18dbe6f3f666c7e40846d2ad79bc47153531e163010feef94d6 WatchSource:0}: Error finding container 9fd733220ff3e18dbe6f3f666c7e40846d2ad79bc47153531e163010feef94d6: Status 404 returned error can't find the container with id 9fd733220ff3e18dbe6f3f666c7e40846d2ad79bc47153531e163010feef94d6 Jul 02 23:02:08.263320 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.263253 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:08.462948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.462920 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:02:08.463122 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.462920 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:02:08.463122 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.462920 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:02:08.465625 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.465605 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 23:02:08.466609 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.466558 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:02:08.466609 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.466558 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9lfbh\"" Jul 02 23:02:08.466766 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.466649 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-llzj2\"" Jul 02 23:02:08.466766 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.466587 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 23:02:08.466766 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.466585 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:02:08.480025 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.480002 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:08.514057 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.514008 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:08.514057 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.514034 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:08.514194 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:08.514167 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:02:08.514249 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:08.514213 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:02:08.514249 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:08.514226 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f6dfc7cc-pjfxq: secret "image-registry-tls" not found Jul 02 23:02:08.514332 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:08.514273 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls podName:759fdb13-b1f4-4476-8a9d-2b52e33105aa nodeName:}" failed. No retries permitted until 2026-07-02 23:02:09.514252494 +0000 UTC m=+36.587504848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls") pod "dns-default-d6kx2" (UID: "759fdb13-b1f4-4476-8a9d-2b52e33105aa") : secret "dns-default-metrics-tls" not found Jul 02 23:02:08.514332 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:08.514292 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls podName:72042bb7-03a7-4133-b397-ecb5c0052cea nodeName:}" failed. No retries permitted until 2026-07-02 23:02:09.514283125 +0000 UTC m=+36.587535472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls") pod "image-registry-84f6dfc7cc-pjfxq" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea") : secret "image-registry-tls" not found Jul 02 23:02:08.670335 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:08.670304 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kdqh7" event={"ID":"54bffe79-6186-4756-a955-f21219245e06","Type":"ContainerStarted","Data":"9fd733220ff3e18dbe6f3f666c7e40846d2ad79bc47153531e163010feef94d6"} Jul 02 23:02:09.423301 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.423274 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:09.467800 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.467761 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jkqkn"] Jul 02 23:02:09.473214 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.473194 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.475643 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.475615 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 23:02:09.475751 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.475677 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 23:02:09.475831 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.475776 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jp64q\"" Jul 02 23:02:09.476810 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.476790 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 23:02:09.476925 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.476847 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 23:02:09.484340 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.484320 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jkqkn"] Jul 02 23:02:09.509662 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.509641 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:09.523605 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.523583 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:09.523722 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.523613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:09.523780 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:09.523738 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:02:09.523833 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:09.523802 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls podName:759fdb13-b1f4-4476-8a9d-2b52e33105aa nodeName:}" failed. No retries permitted until 2026-07-02 23:02:11.523784964 +0000 UTC m=+38.597037306 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls") pod "dns-default-d6kx2" (UID: "759fdb13-b1f4-4476-8a9d-2b52e33105aa") : secret "dns-default-metrics-tls" not found Jul 02 23:02:09.523895 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:09.523742 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:02:09.523895 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:09.523853 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f6dfc7cc-pjfxq: secret "image-registry-tls" not found Jul 02 23:02:09.524000 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:09.523912 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls podName:72042bb7-03a7-4133-b397-ecb5c0052cea nodeName:}" failed. No retries permitted until 2026-07-02 23:02:11.523895702 +0000 UTC m=+38.597148066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls") pod "image-registry-84f6dfc7cc-pjfxq" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea") : secret "image-registry-tls" not found Jul 02 23:02:09.570044 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.570017 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:09.624185 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.624163 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw26h\" (UniqueName: \"kubernetes.io/projected/c3288985-9f79-4260-8ab0-b35085c5c823-kube-api-access-gw26h\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.624315 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.624195 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3288985-9f79-4260-8ab0-b35085c5c823-crio-socket\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.624315 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.624305 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3288985-9f79-4260-8ab0-b35085c5c823-data-volume\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.624404 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.624334 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3288985-9f79-4260-8ab0-b35085c5c823-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.624404 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.624391 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.725541 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.725452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3288985-9f79-4260-8ab0-b35085c5c823-data-volume\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.725541 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.725498 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3288985-9f79-4260-8ab0-b35085c5c823-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.725765 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.725553 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.725765 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.725598 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw26h\" (UniqueName: \"kubernetes.io/projected/c3288985-9f79-4260-8ab0-b35085c5c823-kube-api-access-gw26h\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.725765 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:09.725648 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:02:09.725765 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:09.725717 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3288985-9f79-4260-8ab0-b35085c5c823-kube-rbac-proxy-cm podName:c3288985-9f79-4260-8ab0-b35085c5c823 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:10.225701881 +0000 UTC m=+37.298954244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c3288985-9f79-4260-8ab0-b35085c5c823-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jkqkn" (UID: "c3288985-9f79-4260-8ab0-b35085c5c823") : configmap "kube-rbac-proxy" not found Jul 02 23:02:09.725765 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.725652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3288985-9f79-4260-8ab0-b35085c5c823-crio-socket\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.725971 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:09.725794 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:02:09.725971 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.725807 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3288985-9f79-4260-8ab0-b35085c5c823-crio-socket\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.725971 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:09.725842 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls podName:c3288985-9f79-4260-8ab0-b35085c5c823 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:10.225827243 +0000 UTC m=+37.299079623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jkqkn" (UID: "c3288985-9f79-4260-8ab0-b35085c5c823") : secret "insights-runtime-extractor-tls" not found Jul 02 23:02:09.725971 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.725859 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3288985-9f79-4260-8ab0-b35085c5c823-data-volume\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.734461 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.734439 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw26h\" (UniqueName: \"kubernetes.io/projected/c3288985-9f79-4260-8ab0-b35085c5c823-kube-api-access-gw26h\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:09.812851 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.812830 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:09.912845 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:09.912657 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:10.230839 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:10.230802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3288985-9f79-4260-8ab0-b35085c5c823-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:10.231006 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:10.230862 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:10.231006 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:10.230931 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:02:10.231006 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:10.230972 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:02:10.231006 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:10.231005 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3288985-9f79-4260-8ab0-b35085c5c823-kube-rbac-proxy-cm podName:c3288985-9f79-4260-8ab0-b35085c5c823 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:11.23098517 +0000 UTC m=+38.304237514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c3288985-9f79-4260-8ab0-b35085c5c823-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jkqkn" (UID: "c3288985-9f79-4260-8ab0-b35085c5c823") : configmap "kube-rbac-proxy" not found Jul 02 23:02:10.231231 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:10.231025 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls podName:c3288985-9f79-4260-8ab0-b35085c5c823 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:11.231015536 +0000 UTC m=+38.304267880 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jkqkn" (UID: "c3288985-9f79-4260-8ab0-b35085c5c823") : secret "insights-runtime-extractor-tls" not found Jul 02 23:02:10.288064 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:10.287994 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:10.342184 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:10.342141 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:10.521900 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:10.521860 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:10.625671 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:10.625646 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:10.681068 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:10.681019 2568 generic.go:356] "Generic (PLEG): container finished" podID="ce7afefc-665b-46ab-9895-38facdf8467f" containerID="0e75faae29b8d492acf777f179c653e4d9f769fdcf7d8d564773a816d728a69f" exitCode=0 Jul 02 23:02:10.681230 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:10.681100 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xr5tz" event={"ID":"ce7afefc-665b-46ab-9895-38facdf8467f","Type":"ContainerDied","Data":"0e75faae29b8d492acf777f179c653e4d9f769fdcf7d8d564773a816d728a69f"} Jul 02 23:02:10.720886 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:10.720851 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:10.820116 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:10.820092 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:10.922011 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:10.921937 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:11.239842 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:11.239771 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3288985-9f79-4260-8ab0-b35085c5c823-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:11.239995 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:11.239848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:11.240075 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:11.240012 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:02:11.240119 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:11.240093 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls podName:c3288985-9f79-4260-8ab0-b35085c5c823 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:13.240072618 +0000 UTC m=+40.313324982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jkqkn" (UID: "c3288985-9f79-4260-8ab0-b35085c5c823") : secret "insights-runtime-extractor-tls" not found Jul 02 23:02:11.240343 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:11.240315 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3288985-9f79-4260-8ab0-b35085c5c823-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:11.469497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:11.469469 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4vks2_12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd/dns-node-resolver/0.log" Jul 02 23:02:11.541769 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:11.541689 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:11.541769 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:11.541723 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:11.542407 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:11.541841 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:02:11.542407 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:11.541861 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f6dfc7cc-pjfxq: secret "image-registry-tls" not found Jul 02 23:02:11.542407 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:11.541842 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:02:11.542407 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:11.541918 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls podName:72042bb7-03a7-4133-b397-ecb5c0052cea nodeName:}" failed. No retries permitted until 2026-07-02 23:02:15.541900084 +0000 UTC m=+42.615152444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls") pod "image-registry-84f6dfc7cc-pjfxq" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea") : secret "image-registry-tls" not found Jul 02 23:02:11.542407 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:11.541936 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls podName:759fdb13-b1f4-4476-8a9d-2b52e33105aa nodeName:}" failed. No retries permitted until 2026-07-02 23:02:15.541926494 +0000 UTC m=+42.615178837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls") pod "dns-default-d6kx2" (UID: "759fdb13-b1f4-4476-8a9d-2b52e33105aa") : secret "dns-default-metrics-tls" not found Jul 02 23:02:11.687055 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:11.687021 2568 generic.go:356] "Generic (PLEG): container finished" podID="ce7afefc-665b-46ab-9895-38facdf8467f" containerID="3cd3130bfecb8f868a105dd498e642305b95f774d2eac989b406b453df368921" exitCode=0 Jul 02 23:02:11.687221 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:11.687070 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xr5tz" event={"ID":"ce7afefc-665b-46ab-9895-38facdf8467f","Type":"ContainerDied","Data":"3cd3130bfecb8f868a105dd498e642305b95f774d2eac989b406b453df368921"} Jul 02 23:02:12.160485 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:12.160452 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:12.267947 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:12.267894 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gfdss_ed9789e7-960c-4bb6-b65d-fd51ae63a29e/node-ca/0.log" Jul 02 23:02:12.432966 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:12.432939 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:12.494293 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:12.494260 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:12.691959 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:12.691928 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xr5tz" event={"ID":"ce7afefc-665b-46ab-9895-38facdf8467f","Type":"ContainerStarted","Data":"18a3772b4435ce978b70441c5be6e0390cffe4132e0acf2e2eca0aae64304c6c"} Jul 02 23:02:12.693326 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:12.693306 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kdqh7" event={"ID":"54bffe79-6186-4756-a955-f21219245e06","Type":"ContainerStarted","Data":"ff9f090c090b3a6856316e590a39219a0d5506a608e150b093090a2872f79601"} Jul 02 23:02:12.745705 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:12.745663 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xr5tz" podStartSLOduration=9.942620038 podStartE2EDuration="39.745651136s" totalImagesPullingTime="29.803031098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:12.724724874 +0000 UTC m=+39.797977239" watchObservedRunningTime="2026-07-02 23:02:12.745651136 +0000 UTC m=+39.818903502" Jul 02 23:02:13.052654 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:13.052586 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:13.256751 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:13.256725 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:13.256877 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:13.256855 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:02:13.256917 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:13.256912 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls podName:c3288985-9f79-4260-8ab0-b35085c5c823 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:17.256898106 +0000 UTC m=+44.330150447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jkqkn" (UID: "c3288985-9f79-4260-8ab0-b35085c5c823") : secret "insights-runtime-extractor-tls" not found Jul 02 23:02:13.268834 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:13.268818 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kdqh7_54bffe79-6186-4756-a955-f21219245e06/serve-healthcheck-canary/0.log" Jul 02 23:02:13.830682 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:13.830641 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:13.891818 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:13.891782 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:15.573257 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:15.573216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:15.573257 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:15.573255 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:15.573693 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:15.573379 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:02:15.573693 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:15.573435 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls podName:759fdb13-b1f4-4476-8a9d-2b52e33105aa nodeName:}" failed. No retries permitted until 2026-07-02 23:02:23.573422489 +0000 UTC m=+50.646674846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls") pod "dns-default-d6kx2" (UID: "759fdb13-b1f4-4476-8a9d-2b52e33105aa") : secret "dns-default-metrics-tls" not found Jul 02 23:02:15.573693 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:15.573382 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:02:15.573693 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:15.573477 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f6dfc7cc-pjfxq: secret "image-registry-tls" not found Jul 02 23:02:15.573693 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:15.573519 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls podName:72042bb7-03a7-4133-b397-ecb5c0052cea nodeName:}" failed. No retries permitted until 2026-07-02 23:02:23.573506436 +0000 UTC m=+50.646758779 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls") pod "image-registry-84f6dfc7cc-pjfxq" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea") : secret "image-registry-tls" not found Jul 02 23:02:16.173190 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:16.173158 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:16.815854 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:16.815822 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:17.140972 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:17.140949 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:17.223172 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:17.223150 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:17.286479 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:17.286452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:17.286625 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:17.286607 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:02:17.286671 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:17.286666 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls podName:c3288985-9f79-4260-8ab0-b35085c5c823 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:25.286652598 +0000 UTC m=+52.359904939 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jkqkn" (UID: "c3288985-9f79-4260-8ab0-b35085c5c823") : secret "insights-runtime-extractor-tls" not found Jul 02 23:02:17.320378 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:17.320353 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:17.515661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:17.515603 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:18.323004 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:18.322962 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:18.914077 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:18.914048 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:19.121532 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:19.121500 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:19.361235 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:19.361164 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:19.518852 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:19.516751 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:19.617766 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:19.617723 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:20.447558 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:20.447526 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:20.547937 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:20.547907 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:20.607685 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:20.607656 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:20.916406 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:20.916381 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:21.117137 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:21.117107 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:21.318145 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:21.318071 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:21.373050 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:21.373023 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:21.718835 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:21.718811 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:22.316911 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:22.316882 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:22.713107 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:22.713064 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:23.026720 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.026642 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:23.126379 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.126351 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:23.226948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.226920 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:23.329170 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.329103 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:23.635458 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.635434 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:23.635590 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.635464 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:23.638757 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.638731 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/759fdb13-b1f4-4476-8a9d-2b52e33105aa-metrics-tls\") pod \"dns-default-d6kx2\" (UID: \"759fdb13-b1f4-4476-8a9d-2b52e33105aa\") " pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:23.638998 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.638974 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls\") pod \"image-registry-84f6dfc7cc-pjfxq\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:23.642769 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.642734 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:23.654868 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.654844 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:23.816533 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.816455 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kdqh7" podStartSLOduration=12.707235896 podStartE2EDuration="16.816436032s" totalImagesPullingTime="4.109200136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:12.745292429 +0000 UTC m=+39.818544793" watchObservedRunningTime="2026-07-02 23:02:23.816436032 +0000 UTC m=+50.889688408" Jul 02 23:02:23.816835 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.816810 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84f6dfc7cc-pjfxq"] Jul 02 23:02:23.819659 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:23.819632 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72042bb7_03a7_4133_b397_ecb5c0052cea.slice/crio-4284f4b8e3e8f9e8f087ab201bbb53ba3dd9a920fa48965d24d3271cb2bac960 WatchSource:0}: Error finding container 4284f4b8e3e8f9e8f087ab201bbb53ba3dd9a920fa48965d24d3271cb2bac960: Status 404 returned error can't find the container with id 4284f4b8e3e8f9e8f087ab201bbb53ba3dd9a920fa48965d24d3271cb2bac960 Jul 02 23:02:23.830024 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.830004 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d6kx2"] Jul 02 23:02:23.832707 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:23.832683 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759fdb13_b1f4_4476_8a9d_2b52e33105aa.slice/crio-dbc9b8549bebb48d0b2aa36ef783befdd034ac9350a9e8889206a2c4891b80e3 WatchSource:0}: Error finding container dbc9b8549bebb48d0b2aa36ef783befdd034ac9350a9e8889206a2c4891b80e3: Status 404 returned error can't find the container with id dbc9b8549bebb48d0b2aa36ef783befdd034ac9350a9e8889206a2c4891b80e3 Jul 02 23:02:23.915043 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:23.912423 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:24.320158 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:24.320079 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:24.515859 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:24.515824 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:24.721087 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:24.721051 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d6kx2" event={"ID":"759fdb13-b1f4-4476-8a9d-2b52e33105aa","Type":"ContainerStarted","Data":"dbc9b8549bebb48d0b2aa36ef783befdd034ac9350a9e8889206a2c4891b80e3"} Jul 02 23:02:24.722685 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:24.722657 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" event={"ID":"72042bb7-03a7-4133-b397-ecb5c0052cea","Type":"ContainerStarted","Data":"0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5"} Jul 02 23:02:24.722829 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:24.722703 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" event={"ID":"72042bb7-03a7-4133-b397-ecb5c0052cea","Type":"ContainerStarted","Data":"4284f4b8e3e8f9e8f087ab201bbb53ba3dd9a920fa48965d24d3271cb2bac960"} Jul 02 23:02:24.722829 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:24.722781 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:24.742788 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:24.742745 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" podStartSLOduration=23.742732799 podStartE2EDuration="23.742732799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:02:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:24.742273309 +0000 UTC m=+51.815525672" watchObservedRunningTime="2026-07-02 23:02:24.742732799 +0000 UTC m=+51.815985156" Jul 02 23:02:24.915984 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:24.915944 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:25.348749 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:25.348720 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:25.351044 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:25.351011 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3288985-9f79-4260-8ab0-b35085c5c823-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkqkn\" (UID: \"c3288985-9f79-4260-8ab0-b35085c5c823\") " pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:25.383935 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:25.383912 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jkqkn" Jul 02 23:02:25.516870 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:25.516839 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:25.913242 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:25.913210 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:25.919386 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:25.919362 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jkqkn"] Jul 02 23:02:25.922776 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:25.922751 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3288985_9f79_4260_8ab0_b35085c5c823.slice/crio-afd610372fff3d49a60dc9c7defa6242305b79e0f89540eed9b76c4ee8e296cb WatchSource:0}: Error finding container afd610372fff3d49a60dc9c7defa6242305b79e0f89540eed9b76c4ee8e296cb: Status 404 returned error can't find the container with id afd610372fff3d49a60dc9c7defa6242305b79e0f89540eed9b76c4ee8e296cb Jul 02 23:02:26.310893 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:26.310858 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:26.512423 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:26.512389 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:26.732492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:26.732142 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d6kx2" event={"ID":"759fdb13-b1f4-4476-8a9d-2b52e33105aa","Type":"ContainerStarted","Data":"e29624f3d75cacf51acd20fbb3404f47595f26d0994a931fde48d1a915d3b1e2"} Jul 02 23:02:26.732492 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:26.732206 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d6kx2" event={"ID":"759fdb13-b1f4-4476-8a9d-2b52e33105aa","Type":"ContainerStarted","Data":"822ef04a40b206f8ef402c668c68229c6330e49a645a3c13bbe9e56ac3fb1550"} Jul 02 23:02:26.732724 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:26.732561 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:26.734227 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:26.734202 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkqkn" event={"ID":"c3288985-9f79-4260-8ab0-b35085c5c823","Type":"ContainerStarted","Data":"afd610372fff3d49a60dc9c7defa6242305b79e0f89540eed9b76c4ee8e296cb"} Jul 02 23:02:26.755023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:26.754947 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-d6kx2" podStartSLOduration=17.435312605 podStartE2EDuration="19.754928121s" totalImagesPullingTime="2.319615516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:26.75295748 +0000 UTC m=+53.826209844" watchObservedRunningTime="2026-07-02 23:02:26.754928121 +0000 UTC m=+53.828180487" Jul 02 23:02:27.117828 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:27.117798 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:27.319374 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:27.319336 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:27.516913 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:27.516830 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:29.748007 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:29.747977 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkqkn" event={"ID":"c3288985-9f79-4260-8ab0-b35085c5c823","Type":"ContainerStarted","Data":"df54cc46ea750cb02b905b24a692507a9740d305352d18d8ccf46c344141429a"} Jul 02 23:02:30.753385 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:30.753352 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkqkn" event={"ID":"c3288985-9f79-4260-8ab0-b35085c5c823","Type":"ContainerStarted","Data":"33dbdce11615e2a67188a4073c0f555f5e12dba1c72fd3bc6af3c330fc3d76af"} Jul 02 23:02:32.664734 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.664697 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz"] Jul 02 23:02:32.673947 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.673928 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99"] Jul 02 23:02:32.674082 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.674052 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:32.677188 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.677139 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99" Jul 02 23:02:32.678608 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.678585 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 02 23:02:32.678741 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.678621 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 23:02:32.678741 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.678585 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 23:02:32.679244 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.679223 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 23:02:32.679776 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.679733 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-wwwtz\"" Jul 02 23:02:32.680000 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.679984 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 02 23:02:32.692398 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.692381 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz"] Jul 02 23:02:32.696264 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.696245 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99"] Jul 02 23:02:32.702364 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.702349 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84f6dfc7cc-pjfxq"] Jul 02 23:02:32.702624 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.702599 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkhrb\" (UniqueName: \"kubernetes.io/projected/c95bf8ee-894a-4253-8c01-67d6502f1396-kube-api-access-tkhrb\") pod \"klusterlet-addon-workmgr-84cf9bcd54-lzgsz\" (UID: \"c95bf8ee-894a-4253-8c01-67d6502f1396\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:32.702795 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.702772 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c95bf8ee-894a-4253-8c01-67d6502f1396-tmp\") pod \"klusterlet-addon-workmgr-84cf9bcd54-lzgsz\" (UID: \"c95bf8ee-894a-4253-8c01-67d6502f1396\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:32.702901 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.702818 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c95bf8ee-894a-4253-8c01-67d6502f1396-klusterlet-config\") pod \"klusterlet-addon-workmgr-84cf9bcd54-lzgsz\" (UID: \"c95bf8ee-894a-4253-8c01-67d6502f1396\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:32.702901 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.702845 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ps67\" (UniqueName: \"kubernetes.io/projected/76263e8d-8204-4251-9d5e-2b40098edb7f-kube-api-access-7ps67\") pod \"managed-serviceaccount-addon-agent-58cbf747d6-8db99\" (UID: \"76263e8d-8204-4251-9d5e-2b40098edb7f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99" Jul 02 23:02:32.702996 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.702928 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76263e8d-8204-4251-9d5e-2b40098edb7f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58cbf747d6-8db99\" (UID: \"76263e8d-8204-4251-9d5e-2b40098edb7f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99" Jul 02 23:02:32.757531 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.757513 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55fbb5d77-ff26t"] Jul 02 23:02:32.759636 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.759620 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.782867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.782845 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55fbb5d77-ff26t"] Jul 02 23:02:32.795834 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.795812 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:32.803828 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.803810 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9246c550-93af-4a26-9efb-2af46970ccdb-registry-certificates\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.803932 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.803863 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9246c550-93af-4a26-9efb-2af46970ccdb-trusted-ca\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.803932 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.803887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkhrb\" (UniqueName: \"kubernetes.io/projected/c95bf8ee-894a-4253-8c01-67d6502f1396-kube-api-access-tkhrb\") pod \"klusterlet-addon-workmgr-84cf9bcd54-lzgsz\" (UID: \"c95bf8ee-894a-4253-8c01-67d6502f1396\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:32.803932 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.803902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c95bf8ee-894a-4253-8c01-67d6502f1396-tmp\") pod \"klusterlet-addon-workmgr-84cf9bcd54-lzgsz\" (UID: \"c95bf8ee-894a-4253-8c01-67d6502f1396\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:32.803932 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.803918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9246c550-93af-4a26-9efb-2af46970ccdb-registry-tls\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.804065 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.803934 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9246c550-93af-4a26-9efb-2af46970ccdb-bound-sa-token\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.804065 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.803947 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcjxg\" (UniqueName: \"kubernetes.io/projected/9246c550-93af-4a26-9efb-2af46970ccdb-kube-api-access-hcjxg\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.804065 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.803964 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9246c550-93af-4a26-9efb-2af46970ccdb-installation-pull-secrets\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.804172 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.804065 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9246c550-93af-4a26-9efb-2af46970ccdb-image-registry-private-configuration\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.804172 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.804108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76263e8d-8204-4251-9d5e-2b40098edb7f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58cbf747d6-8db99\" (UID: \"76263e8d-8204-4251-9d5e-2b40098edb7f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99" Jul 02 23:02:32.804172 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.804162 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ps67\" (UniqueName: \"kubernetes.io/projected/76263e8d-8204-4251-9d5e-2b40098edb7f-kube-api-access-7ps67\") pod \"managed-serviceaccount-addon-agent-58cbf747d6-8db99\" (UID: \"76263e8d-8204-4251-9d5e-2b40098edb7f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99" Jul 02 23:02:32.804284 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.804197 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c95bf8ee-894a-4253-8c01-67d6502f1396-klusterlet-config\") pod \"klusterlet-addon-workmgr-84cf9bcd54-lzgsz\" (UID: \"c95bf8ee-894a-4253-8c01-67d6502f1396\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:32.804284 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.804224 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9246c550-93af-4a26-9efb-2af46970ccdb-ca-trust-extracted\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.804352 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.804285 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c95bf8ee-894a-4253-8c01-67d6502f1396-tmp\") pod \"klusterlet-addon-workmgr-84cf9bcd54-lzgsz\" (UID: \"c95bf8ee-894a-4253-8c01-67d6502f1396\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:32.806429 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.806405 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c95bf8ee-894a-4253-8c01-67d6502f1396-klusterlet-config\") pod \"klusterlet-addon-workmgr-84cf9bcd54-lzgsz\" (UID: \"c95bf8ee-894a-4253-8c01-67d6502f1396\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:32.806530 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.806427 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76263e8d-8204-4251-9d5e-2b40098edb7f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58cbf747d6-8db99\" (UID: \"76263e8d-8204-4251-9d5e-2b40098edb7f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99" Jul 02 23:02:32.814558 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.814540 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ps67\" (UniqueName: \"kubernetes.io/projected/76263e8d-8204-4251-9d5e-2b40098edb7f-kube-api-access-7ps67\") pod \"managed-serviceaccount-addon-agent-58cbf747d6-8db99\" (UID: \"76263e8d-8204-4251-9d5e-2b40098edb7f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99" Jul 02 23:02:32.815946 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.815930 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkhrb\" (UniqueName: \"kubernetes.io/projected/c95bf8ee-894a-4253-8c01-67d6502f1396-kube-api-access-tkhrb\") pod \"klusterlet-addon-workmgr-84cf9bcd54-lzgsz\" (UID: \"c95bf8ee-894a-4253-8c01-67d6502f1396\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:32.885323 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.885298 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:32.904850 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.904828 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9246c550-93af-4a26-9efb-2af46970ccdb-installation-pull-secrets\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.904922 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.904858 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9246c550-93af-4a26-9efb-2af46970ccdb-image-registry-private-configuration\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.904922 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.904898 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9246c550-93af-4a26-9efb-2af46970ccdb-ca-trust-extracted\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.904922 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.904917 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9246c550-93af-4a26-9efb-2af46970ccdb-registry-certificates\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.905046 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.904964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9246c550-93af-4a26-9efb-2af46970ccdb-trusted-ca\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.905099 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.905077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9246c550-93af-4a26-9efb-2af46970ccdb-registry-tls\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.905149 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.905119 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9246c550-93af-4a26-9efb-2af46970ccdb-bound-sa-token\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.905210 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.905146 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcjxg\" (UniqueName: \"kubernetes.io/projected/9246c550-93af-4a26-9efb-2af46970ccdb-kube-api-access-hcjxg\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.905558 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.905530 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9246c550-93af-4a26-9efb-2af46970ccdb-ca-trust-extracted\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.905923 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.905902 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9246c550-93af-4a26-9efb-2af46970ccdb-registry-certificates\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.906321 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.906296 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9246c550-93af-4a26-9efb-2af46970ccdb-trusted-ca\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.907062 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.907034 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9246c550-93af-4a26-9efb-2af46970ccdb-installation-pull-secrets\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.907177 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.907162 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9246c550-93af-4a26-9efb-2af46970ccdb-image-registry-private-configuration\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.907377 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.907360 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9246c550-93af-4a26-9efb-2af46970ccdb-registry-tls\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.917605 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.917542 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcjxg\" (UniqueName: \"kubernetes.io/projected/9246c550-93af-4a26-9efb-2af46970ccdb-kube-api-access-hcjxg\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.917769 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.917753 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9246c550-93af-4a26-9efb-2af46970ccdb-bound-sa-token\") pod \"image-registry-55fbb5d77-ff26t\" (UID: \"9246c550-93af-4a26-9efb-2af46970ccdb\") " pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:32.984654 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.984634 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:32.985514 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:32.985490 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:33.000650 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.000621 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99" Jul 02 23:02:33.068182 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.067903 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:33.085545 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.085408 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:33.111812 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.111780 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz"] Jul 02 23:02:33.114308 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:33.114278 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95bf8ee_894a_4253_8c01_67d6502f1396.slice/crio-d7c3f9d0c5e4affb23f00e5e2023491420a5f61435ad900878a7214bb68912cd WatchSource:0}: Error finding container d7c3f9d0c5e4affb23f00e5e2023491420a5f61435ad900878a7214bb68912cd: Status 404 returned error can't find the container with id d7c3f9d0c5e4affb23f00e5e2023491420a5f61435ad900878a7214bb68912cd Jul 02 23:02:33.128341 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.128313 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99"] Jul 02 23:02:33.132107 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:33.132078 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76263e8d_8204_4251_9d5e_2b40098edb7f.slice/crio-f4eba1a1a4aee26e4778dd09e7d708f8504704a1cbf4bfcbd455a0f7adc9d390 WatchSource:0}: Error finding container f4eba1a1a4aee26e4778dd09e7d708f8504704a1cbf4bfcbd455a0f7adc9d390: Status 404 returned error can't find the container with id f4eba1a1a4aee26e4778dd09e7d708f8504704a1cbf4bfcbd455a0f7adc9d390 Jul 02 23:02:33.193082 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.193018 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55fbb5d77-ff26t"] Jul 02 23:02:33.195981 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:33.195957 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9246c550_93af_4a26_9efb_2af46970ccdb.slice/crio-ae1b41035d985f1cc64340621f3199618ee073ce73006e823bcce705c26cbe26 WatchSource:0}: Error finding container ae1b41035d985f1cc64340621f3199618ee073ce73006e823bcce705c26cbe26: Status 404 returned error can't find the container with id ae1b41035d985f1cc64340621f3199618ee073ce73006e823bcce705c26cbe26 Jul 02 23:02:33.763952 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.763914 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" event={"ID":"9246c550-93af-4a26-9efb-2af46970ccdb","Type":"ContainerStarted","Data":"c2d3a85cbcfd8734562cab42c3851f456005e7a28981531dd7417b8bb387ff56"} Jul 02 23:02:33.764361 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.763967 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" event={"ID":"9246c550-93af-4a26-9efb-2af46970ccdb","Type":"ContainerStarted","Data":"ae1b41035d985f1cc64340621f3199618ee073ce73006e823bcce705c26cbe26"} Jul 02 23:02:33.764361 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.764042 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:33.765161 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.765134 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99" event={"ID":"76263e8d-8204-4251-9d5e-2b40098edb7f","Type":"ContainerStarted","Data":"f4eba1a1a4aee26e4778dd09e7d708f8504704a1cbf4bfcbd455a0f7adc9d390"} Jul 02 23:02:33.766945 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.766923 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkqkn" event={"ID":"c3288985-9f79-4260-8ab0-b35085c5c823","Type":"ContainerStarted","Data":"38d919add6bf9751e0abe4e00b77aa5e817f2ae5acfd6d214fdcbe1e07fc3fbc"} Jul 02 23:02:33.767984 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.767965 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" event={"ID":"c95bf8ee-894a-4253-8c01-67d6502f1396","Type":"ContainerStarted","Data":"d7c3f9d0c5e4affb23f00e5e2023491420a5f61435ad900878a7214bb68912cd"} Jul 02 23:02:33.783510 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.783468 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" podStartSLOduration=1.783457592 podStartE2EDuration="1.783457592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:33.78279206 +0000 UTC m=+60.856044425" watchObservedRunningTime="2026-07-02 23:02:33.783457592 +0000 UTC m=+60.856709955" Jul 02 23:02:33.799481 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.799439 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jkqkn" podStartSLOduration=18.119743575 podStartE2EDuration="24.799426501s" totalImagesPullingTime="6.679682926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:02:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:33.798494402 +0000 UTC m=+60.871746762" watchObservedRunningTime="2026-07-02 23:02:33.799426501 +0000 UTC m=+60.872678893" Jul 02 23:02:33.963380 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:33.963328 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:34.019649 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:34.019559 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:34.265419 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:34.265377 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:34.322052 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:34.321973 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:34.677626 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:34.676972 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmwh5" Jul 02 23:02:35.271501 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:35.271465 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:35.329060 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:35.329032 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:35.670863 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:35.670809 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:35.768030 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:35.768002 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:36.675745 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:36.675711 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:36.742442 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:36.742418 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-d6kx2" Jul 02 23:02:36.779210 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:36.779176 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99" event={"ID":"76263e8d-8204-4251-9d5e-2b40098edb7f","Type":"ContainerStarted","Data":"a137929e96accc37460738e6ef050be36e4f6fdf4c362ab4f5f35d425d0b64b3"} Jul 02 23:02:36.795739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:36.795698 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58cbf747d6-8db99" podStartSLOduration=2.138854388 podStartE2EDuration="4.795683412s" totalImagesPullingTime="2.656829024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:02:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:36.795434759 +0000 UTC m=+63.868687124" watchObservedRunningTime="2026-07-02 23:02:36.795683412 +0000 UTC m=+63.868935765" Jul 02 23:02:36.868594 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:36.868542 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:37.027638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:37.027578 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:37.122229 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:37.122208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:37.222100 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:37.222079 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:37.282789 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:37.282734 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:38.164704 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:38.164675 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:38.221559 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:38.221535 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:39.152822 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.152787 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:02:39.153011 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.152885 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:02:39.155476 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.155456 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:02:39.156467 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.156453 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:02:39.165394 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.165364 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb3efb95-e804-4cae-b3b5-220db506cfe8-original-pull-secret\") pod \"global-pull-secret-syncer-lnsbx\" (UID: \"bb3efb95-e804-4cae-b3b5-220db506cfe8\") " pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:02:39.165683 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.165459 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bad930f0-e377-445a-88a5-7e81b222ce9b-metrics-certs\") pod \"network-metrics-daemon-mm52v\" (UID: \"bad930f0-e377-445a-88a5-7e81b222ce9b\") " pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:02:39.353790 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.353753 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqtqc\" (UniqueName: \"kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc\") pod \"network-check-target-hzb5r\" (UID: \"cc68e25f-5e72-4518-9f6e-b7b1870bdf97\") " pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:02:39.356448 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.356429 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 23:02:39.366330 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.366311 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 23:02:39.376623 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.376605 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqtqc\" (UniqueName: \"kubernetes.io/projected/cc68e25f-5e72-4518-9f6e-b7b1870bdf97-kube-api-access-kqtqc\") pod \"network-check-target-hzb5r\" (UID: \"cc68e25f-5e72-4518-9f6e-b7b1870bdf97\") " pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:02:39.383732 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.383715 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-llzj2\"" Jul 02 23:02:39.385900 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.385888 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lnsbx" Jul 02 23:02:39.392330 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.392306 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mm52v" Jul 02 23:02:39.524755 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.524726 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lnsbx"] Jul 02 23:02:39.527729 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:39.527704 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3efb95_e804_4cae_b3b5_220db506cfe8.slice/crio-1d51dc65dc343508e1237860abde1479bcdea1bdfd4031abef025df5c9bb513f WatchSource:0}: Error finding container 1d51dc65dc343508e1237860abde1479bcdea1bdfd4031abef025df5c9bb513f: Status 404 returned error can't find the container with id 1d51dc65dc343508e1237860abde1479bcdea1bdfd4031abef025df5c9bb513f Jul 02 23:02:39.544774 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.544751 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mm52v"] Jul 02 23:02:39.547710 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:39.547689 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad930f0_e377_445a_88a5_7e81b222ce9b.slice/crio-fd93969840ed11273943cef4c2e8ca3ee5814fde22778c7382d4eff333b5c287 WatchSource:0}: Error finding container fd93969840ed11273943cef4c2e8ca3ee5814fde22778c7382d4eff333b5c287: Status 404 returned error can't find the container with id fd93969840ed11273943cef4c2e8ca3ee5814fde22778c7382d4eff333b5c287 Jul 02 23:02:39.676312 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.676289 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9lfbh\"" Jul 02 23:02:39.684394 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.684353 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:02:39.791303 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.791267 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mm52v" event={"ID":"bad930f0-e377-445a-88a5-7e81b222ce9b","Type":"ContainerStarted","Data":"fd93969840ed11273943cef4c2e8ca3ee5814fde22778c7382d4eff333b5c287"} Jul 02 23:02:39.792666 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.792636 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lnsbx" event={"ID":"bb3efb95-e804-4cae-b3b5-220db506cfe8","Type":"ContainerStarted","Data":"1d51dc65dc343508e1237860abde1479bcdea1bdfd4031abef025df5c9bb513f"} Jul 02 23:02:39.807727 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:39.807702 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hzb5r"] Jul 02 23:02:39.810919 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:39.810895 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc68e25f_5e72_4518_9f6e_b7b1870bdf97.slice/crio-541c87e09c95a848ec285e08a0f5945bfa0139ecab483e3457c1c46545811a1a WatchSource:0}: Error finding container 541c87e09c95a848ec285e08a0f5945bfa0139ecab483e3457c1c46545811a1a: Status 404 returned error can't find the container with id 541c87e09c95a848ec285e08a0f5945bfa0139ecab483e3457c1c46545811a1a Jul 02 23:02:40.060626 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:40.060452 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:40.147670 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:40.147637 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:40.314882 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:40.314794 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:40.799108 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:40.799068 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mm52v" event={"ID":"bad930f0-e377-445a-88a5-7e81b222ce9b","Type":"ContainerStarted","Data":"b4e8d2c29368611c456b03fd67d1e1eeae1d2b8b9ed69c14ff3a18ff93827494"} Jul 02 23:02:40.799267 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:40.799218 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mm52v" event={"ID":"bad930f0-e377-445a-88a5-7e81b222ce9b","Type":"ContainerStarted","Data":"85e4bbe38565f02d449006976c59576b26617aa6b5882c295c3aa39ad3c7cdd7"} Jul 02 23:02:40.800791 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:40.800696 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hzb5r" event={"ID":"cc68e25f-5e72-4518-9f6e-b7b1870bdf97","Type":"ContainerStarted","Data":"541c87e09c95a848ec285e08a0f5945bfa0139ecab483e3457c1c46545811a1a"} Jul 02 23:02:40.818370 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:40.818320 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mm52v" podStartSLOduration=66.89187466999999 podStartE2EDuration="1m7.818300487s" totalImagesPullingTime="926.425817ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:40.81653726 +0000 UTC m=+67.889789623" watchObservedRunningTime="2026-07-02 23:02:40.818300487 +0000 UTC m=+67.891552851" Jul 02 23:02:40.904379 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:40.904354 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:40.963603 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:40.963478 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:42.203953 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:42.203881 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:42.709282 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:42.709240 2568 patch_prober.go:28] interesting pod/image-registry-84f6dfc7cc-pjfxq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 23:02:42.709447 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:42.709307 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" podUID="72042bb7-03a7-4133-b397-ecb5c0052cea" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 23:02:43.065499 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:43.065418 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:43.121776 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:43.121739 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:44.223946 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.223912 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:44.444177 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.444122 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:44.470811 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.470783 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb"] Jul 02 23:02:44.472845 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.472829 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.475780 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.475591 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 23:02:44.475780 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.475625 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 23:02:44.477974 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.477955 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xdcq8\"" Jul 02 23:02:44.478071 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.478019 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 02 23:02:44.478219 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.478199 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 02 23:02:44.478314 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.478260 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 23:02:44.478377 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.478359 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ghk6j"] Jul 02 23:02:44.480361 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.480340 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.482533 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.482512 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 23:02:44.482649 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.482561 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 23:02:44.482649 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.482618 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 23:02:44.482649 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.482631 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-25f5f\"" Jul 02 23:02:44.487022 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.487004 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb"] Jul 02 23:02:44.495036 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.495018 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2"] Jul 02 23:02:44.497263 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.497245 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.500121 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.500030 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 02 23:02:44.500198 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.500160 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 02 23:02:44.500198 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.500183 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 02 23:02:44.500267 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.500222 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mtxxf\"" Jul 02 23:02:44.510471 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.510450 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2"] Jul 02 23:02:44.545748 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.545721 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:44.593464 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593435 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-accelerators-collector-config\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.593590 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593480 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23521425-31dd-4d23-90ce-e855a6b5549b-sys\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.593590 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593508 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.593702 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593590 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.593702 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593630 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.593702 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593668 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-tls\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.593702 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593697 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.593857 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593723 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.593857 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593751 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7583ca97-2e44-462c-b17c-cc79dc6f8589-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.593857 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593781 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7583ca97-2e44-462c-b17c-cc79dc6f8589-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.593857 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593811 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7583ca97-2e44-462c-b17c-cc79dc6f8589-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.593857 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593836 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-textfile\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.594064 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593875 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/23521425-31dd-4d23-90ce-e855a6b5549b-root\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.594064 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6t6l\" (UniqueName: \"kubernetes.io/projected/23521425-31dd-4d23-90ce-e855a6b5549b-kube-api-access-p6t6l\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.594064 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.594064 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593939 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4rpd\" (UniqueName: \"kubernetes.io/projected/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-api-access-v4rpd\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.594064 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.593960 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-wtmp\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.594064 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.594012 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23521425-31dd-4d23-90ce-e855a6b5549b-metrics-client-ca\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.594064 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.594046 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dbqh\" (UniqueName: \"kubernetes.io/projected/7583ca97-2e44-462c-b17c-cc79dc6f8589-kube-api-access-2dbqh\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.644663 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.644639 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:44.695334 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695308 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23521425-31dd-4d23-90ce-e855a6b5549b-sys\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.695476 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.695476 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695380 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.695476 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695443 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23521425-31dd-4d23-90ce-e855a6b5549b-sys\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.695657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695488 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.695657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695533 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-tls\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.695657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695562 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.695657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695605 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.695657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695634 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7583ca97-2e44-462c-b17c-cc79dc6f8589-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.696023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695671 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7583ca97-2e44-462c-b17c-cc79dc6f8589-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.696023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695720 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7583ca97-2e44-462c-b17c-cc79dc6f8589-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.696023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695755 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-textfile\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.696023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695782 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/23521425-31dd-4d23-90ce-e855a6b5549b-root\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.696023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695806 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6t6l\" (UniqueName: \"kubernetes.io/projected/23521425-31dd-4d23-90ce-e855a6b5549b-kube-api-access-p6t6l\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.696023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695832 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.696023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695856 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4rpd\" (UniqueName: \"kubernetes.io/projected/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-api-access-v4rpd\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.696023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-wtmp\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.696023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695911 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23521425-31dd-4d23-90ce-e855a6b5549b-metrics-client-ca\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.696023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695941 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dbqh\" (UniqueName: \"kubernetes.io/projected/7583ca97-2e44-462c-b17c-cc79dc6f8589-kube-api-access-2dbqh\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.696023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.695973 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-accelerators-collector-config\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.696588 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.696114 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.696588 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:44.696216 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 23:02:44.696588 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:44.696277 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-tls podName:23521425-31dd-4d23-90ce-e855a6b5549b nodeName:}" failed. No retries permitted until 2026-07-02 23:02:45.196257189 +0000 UTC m=+72.269509543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-tls") pod "node-exporter-ghk6j" (UID: "23521425-31dd-4d23-90ce-e855a6b5549b") : secret "node-exporter-tls" not found Jul 02 23:02:44.696588 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.696500 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7583ca97-2e44-462c-b17c-cc79dc6f8589-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.696588 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.696524 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-accelerators-collector-config\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.697903 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.697114 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.697903 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:44.697234 2568 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 02 23:02:44.697903 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:44.697300 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7583ca97-2e44-462c-b17c-cc79dc6f8589-openshift-state-metrics-tls podName:7583ca97-2e44-462c-b17c-cc79dc6f8589 nodeName:}" failed. No retries permitted until 2026-07-02 23:02:45.197281421 +0000 UTC m=+72.270533767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7583ca97-2e44-462c-b17c-cc79dc6f8589-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-hpdzb" (UID: "7583ca97-2e44-462c-b17c-cc79dc6f8589") : secret "openshift-state-metrics-tls" not found Jul 02 23:02:44.697903 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.697538 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.697903 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.697543 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-textfile\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.697903 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.697621 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/23521425-31dd-4d23-90ce-e855a6b5549b-root\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.697903 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.697643 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-wtmp\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.698376 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.698008 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23521425-31dd-4d23-90ce-e855a6b5549b-metrics-client-ca\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.699307 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.699285 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.699485 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.699439 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7583ca97-2e44-462c-b17c-cc79dc6f8589-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.699823 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.699783 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.699914 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.699849 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.708238 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.708195 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4rpd\" (UniqueName: \"kubernetes.io/projected/095e97d7-a7b1-437e-a1e2-dadc8524eaf9-kube-api-access-v4rpd\") pod \"kube-state-metrics-7c9dbb8b7d-4lmt2\" (UID: \"095e97d7-a7b1-437e-a1e2-dadc8524eaf9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.709934 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.709886 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6t6l\" (UniqueName: \"kubernetes.io/projected/23521425-31dd-4d23-90ce-e855a6b5549b-kube-api-access-p6t6l\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:44.710687 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.710670 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dbqh\" (UniqueName: \"kubernetes.io/projected/7583ca97-2e44-462c-b17c-cc79dc6f8589-kube-api-access-2dbqh\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:44.745554 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.745503 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:44.808369 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.808345 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" Jul 02 23:02:44.844928 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.844906 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:44.946077 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:44.946043 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:45.200716 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.200684 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-tls\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:45.200897 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.200745 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7583ca97-2e44-462c-b17c-cc79dc6f8589-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:45.200897 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:45.200846 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 23:02:45.201009 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:02:45.200920 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-tls podName:23521425-31dd-4d23-90ce-e855a6b5549b nodeName:}" failed. No retries permitted until 2026-07-02 23:02:46.200901572 +0000 UTC m=+73.274153935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-tls") pod "node-exporter-ghk6j" (UID: "23521425-31dd-4d23-90ce-e855a6b5549b") : secret "node-exporter-tls" not found Jul 02 23:02:45.203331 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.203308 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7583ca97-2e44-462c-b17c-cc79dc6f8589-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hpdzb\" (UID: \"7583ca97-2e44-462c-b17c-cc79dc6f8589\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:45.384290 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.384258 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" Jul 02 23:02:45.415028 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.415002 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:45.515350 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.515262 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:45.545975 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.545951 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:02:45.548773 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.548755 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.552657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.551427 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 23:02:45.552657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.551678 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 23:02:45.552657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.551875 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 23:02:45.552657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.552031 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 23:02:45.552657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.552226 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 23:02:45.552657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.552234 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 23:02:45.552657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.552039 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 23:02:45.552657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.552416 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 23:02:45.552657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.552445 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 23:02:45.552657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.552536 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wz6tb\"" Jul 02 23:02:45.564805 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.564744 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:02:45.596968 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.596420 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606255 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-config-volume\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606297 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-tls-assets\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606330 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdmqw\" (UniqueName: \"kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-kube-api-access-bdmqw\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606403 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606469 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-web-config\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606514 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606546 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606632 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606655 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606690 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606750 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-config-out\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.606867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.606783 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.698395 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.698352 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb"] Jul 02 23:02:45.700984 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.700908 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:45.703460 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:45.703213 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7583ca97_2e44_462c_b17c_cc79dc6f8589.slice/crio-e3a7cf9ddfb459b1ca0de50ce41686c7374f597ff326328f644dfd936f483d5f WatchSource:0}: Error finding container e3a7cf9ddfb459b1ca0de50ce41686c7374f597ff326328f644dfd936f483d5f: Status 404 returned error can't find the container with id e3a7cf9ddfb459b1ca0de50ce41686c7374f597ff326328f644dfd936f483d5f Jul 02 23:02:45.707618 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707594 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-config-out\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.707704 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707632 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.707704 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707663 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-config-volume\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.707704 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707688 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-tls-assets\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.707796 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707713 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdmqw\" (UniqueName: \"kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-kube-api-access-bdmqw\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.707796 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707743 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.707796 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707773 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.707880 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707834 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-web-config\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.707909 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707876 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.707943 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707908 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.707976 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.708007 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.707975 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.708035 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.708007 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.709019 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.708948 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.709102 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.709073 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.709375 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.709197 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.712952 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.712903 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-config-out\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.713852 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.713809 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-web-config\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.714423 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.714354 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.714423 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.714366 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.714806 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.714781 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.715090 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.715070 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.716075 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.716004 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.718097 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.717983 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-config-volume\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.718365 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.718333 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-tls-assets\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.718500 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.718475 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2"] Jul 02 23:02:45.720137 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.720089 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdmqw\" (UniqueName: \"kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-kube-api-access-bdmqw\") pod \"alertmanager-main-0\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.723750 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:45.723724 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095e97d7_a7b1_437e_a1e2_dadc8524eaf9.slice/crio-17db346ffbcbe7cdd005e1869440e08928be3acf264b6cf2661b86bb6778e8f8 WatchSource:0}: Error finding container 17db346ffbcbe7cdd005e1869440e08928be3acf264b6cf2661b86bb6778e8f8: Status 404 returned error can't find the container with id 17db346ffbcbe7cdd005e1869440e08928be3acf264b6cf2661b86bb6778e8f8 Jul 02 23:02:45.825109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.825069 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" event={"ID":"c95bf8ee-894a-4253-8c01-67d6502f1396","Type":"ContainerStarted","Data":"4fbf362b49469651eabd2b0addf20c2365bdd77c59d30c93d7b243549030f3dc"} Jul 02 23:02:45.825351 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.825299 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:45.826937 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.826905 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hzb5r" event={"ID":"cc68e25f-5e72-4518-9f6e-b7b1870bdf97","Type":"ContainerStarted","Data":"cadb4e609f3ea592ca6f8584ea03c87b83fc24f3c18b2c3957e3bb24980720b4"} Jul 02 23:02:45.827040 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.826956 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:02:45.827155 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.827133 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" Jul 02 23:02:45.827992 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.827972 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" event={"ID":"095e97d7-a7b1-437e-a1e2-dadc8524eaf9","Type":"ContainerStarted","Data":"17db346ffbcbe7cdd005e1869440e08928be3acf264b6cf2661b86bb6778e8f8"} Jul 02 23:02:45.830046 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.830005 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" event={"ID":"7583ca97-2e44-462c-b17c-cc79dc6f8589","Type":"ContainerStarted","Data":"3b52dbbd3eb8ed278f96849857e490a82818418040dd3f6669cf0e4beaf962c4"} Jul 02 23:02:45.830133 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.830074 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" event={"ID":"7583ca97-2e44-462c-b17c-cc79dc6f8589","Type":"ContainerStarted","Data":"e3a7cf9ddfb459b1ca0de50ce41686c7374f597ff326328f644dfd936f483d5f"} Jul 02 23:02:45.831515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.831456 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lnsbx" event={"ID":"bb3efb95-e804-4cae-b3b5-220db506cfe8","Type":"ContainerStarted","Data":"fe503eecf251a80c2d58b10daf5584fd683b432c9d0fb57ea776db8b4c839475"} Jul 02 23:02:45.842117 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.842076 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84cf9bcd54-lzgsz" podStartSLOduration=1.422956207 podStartE2EDuration="13.842064066s" totalImagesPullingTime="12.419107859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:02:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:45.840662179 +0000 UTC m=+72.913914542" watchObservedRunningTime="2026-07-02 23:02:45.842064066 +0000 UTC m=+72.915316428" Jul 02 23:02:45.862021 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.861995 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:02:45.866206 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.866181 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:45.875216 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.875154 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lnsbx" podStartSLOduration=66.859766303 podStartE2EDuration="1m12.875140573s" totalImagesPullingTime="6.01537427s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:45.874722407 +0000 UTC m=+72.947974770" watchObservedRunningTime="2026-07-02 23:02:45.875140573 +0000 UTC m=+72.948392913" Jul 02 23:02:45.889055 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.889006 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hzb5r" podStartSLOduration=67.15607048 podStartE2EDuration="1m12.888990591s" totalImagesPullingTime="5.732920111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:45.888751122 +0000 UTC m=+72.962003486" watchObservedRunningTime="2026-07-02 23:02:45.888990591 +0000 UTC m=+72.962242954" Jul 02 23:02:45.952060 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.952031 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:45.995466 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:45.995438 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:02:45.998727 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:45.998701 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b1a49b_041c_4788_84b1_71104153a384.slice/crio-22976886022cc520a9ee7648dc0e562452b8b0e4f7701cdb51b5363a0b6137b1 WatchSource:0}: Error finding container 22976886022cc520a9ee7648dc0e562452b8b0e4f7701cdb51b5363a0b6137b1: Status 404 returned error can't find the container with id 22976886022cc520a9ee7648dc0e562452b8b0e4f7701cdb51b5363a0b6137b1 Jul 02 23:02:46.212900 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.212870 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-tls\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:46.215103 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.215081 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23521425-31dd-4d23-90ce-e855a6b5549b-node-exporter-tls\") pod \"node-exporter-ghk6j\" (UID: \"23521425-31dd-4d23-90ce-e855a6b5549b\") " pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:46.293060 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.292966 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ghk6j" Jul 02 23:02:46.302360 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:46.302334 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23521425_31dd_4d23_90ce_e855a6b5549b.slice/crio-afeb0361c834b0ec6ac8fbd4edf064b7032620264ba6962d5efe6a8203c8864e WatchSource:0}: Error finding container afeb0361c834b0ec6ac8fbd4edf064b7032620264ba6962d5efe6a8203c8864e: Status 404 returned error can't find the container with id afeb0361c834b0ec6ac8fbd4edf064b7032620264ba6962d5efe6a8203c8864e Jul 02 23:02:46.824942 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.824902 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-rjnfd"] Jul 02 23:02:46.827351 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.827324 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-rjnfd" Jul 02 23:02:46.830070 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.830051 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-tfgz4\"" Jul 02 23:02:46.830466 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.830451 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 23:02:46.830686 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.830668 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 23:02:46.840384 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.840329 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghk6j" event={"ID":"23521425-31dd-4d23-90ce-e855a6b5549b","Type":"ContainerStarted","Data":"afeb0361c834b0ec6ac8fbd4edf064b7032620264ba6962d5efe6a8203c8864e"} Jul 02 23:02:46.843198 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.843176 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerStarted","Data":"22976886022cc520a9ee7648dc0e562452b8b0e4f7701cdb51b5363a0b6137b1"} Jul 02 23:02:46.843303 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.843246 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-rjnfd"] Jul 02 23:02:46.846426 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.845619 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" event={"ID":"7583ca97-2e44-462c-b17c-cc79dc6f8589","Type":"ContainerStarted","Data":"5f19d975358466c698951409fb0157fdf0d7c1335bef364ab22cdcd3922da765"} Jul 02 23:02:46.849859 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.849823 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:46.920847 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.920817 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhdh9\" (UniqueName: \"kubernetes.io/projected/cfbde320-130d-4761-81df-3154c7c0f7e2-kube-api-access-bhdh9\") pod \"downloads-685bfb5f64-rjnfd\" (UID: \"cfbde320-130d-4761-81df-3154c7c0f7e2\") " pod="openshift-console/downloads-685bfb5f64-rjnfd" Jul 02 23:02:46.946164 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:46.946138 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:47.021754 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.021727 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhdh9\" (UniqueName: \"kubernetes.io/projected/cfbde320-130d-4761-81df-3154c7c0f7e2-kube-api-access-bhdh9\") pod \"downloads-685bfb5f64-rjnfd\" (UID: \"cfbde320-130d-4761-81df-3154c7c0f7e2\") " pod="openshift-console/downloads-685bfb5f64-rjnfd" Jul 02 23:02:47.032740 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.032716 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhdh9\" (UniqueName: \"kubernetes.io/projected/cfbde320-130d-4761-81df-3154c7c0f7e2-kube-api-access-bhdh9\") pod \"downloads-685bfb5f64-rjnfd\" (UID: \"cfbde320-130d-4761-81df-3154c7c0f7e2\") " pod="openshift-console/downloads-685bfb5f64-rjnfd" Jul 02 23:02:47.048145 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.048120 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:47.139592 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.139552 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-rjnfd" Jul 02 23:02:47.167130 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.167103 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:47.312978 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.312942 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:47.451883 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.451783 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:47.552818 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.552779 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:47.656718 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.656692 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:47.853953 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.853924 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:47.955802 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.955710 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:47.982929 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:47.982903 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-rjnfd"] Jul 02 23:02:47.986243 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:47.986212 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbde320_130d_4761_81df_3154c7c0f7e2.slice/crio-dd16617929c3e195b333a480e142627533471059943f713de9635d8856fe5244 WatchSource:0}: Error finding container dd16617929c3e195b333a480e142627533471059943f713de9635d8856fe5244: Status 404 returned error can't find the container with id dd16617929c3e195b333a480e142627533471059943f713de9635d8856fe5244 Jul 02 23:02:48.523635 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.523596 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:48.719993 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.719948 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:48.773500 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.773471 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-67dcb868c9-w2hpl"] Jul 02 23:02:48.780195 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.780140 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.783192 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.783167 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 02 23:02:48.784353 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.784079 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 23:02:48.784353 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.784090 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 02 23:02:48.784353 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.784139 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ctp4w\"" Jul 02 23:02:48.784353 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.784090 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ec5rk5f79mead\"" Jul 02 23:02:48.784353 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.784322 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 02 23:02:48.786183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.786158 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67dcb868c9-w2hpl"] Jul 02 23:02:48.800966 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.800931 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:48.840884 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.840860 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdfpq\" (UniqueName: \"kubernetes.io/projected/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-kube-api-access-rdfpq\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.840977 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.840904 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.840977 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.840931 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-metrics-server-audit-profiles\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.841104 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.841005 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-secret-metrics-server-client-certs\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.841104 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.841036 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-secret-metrics-server-tls\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.841104 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.841089 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-client-ca-bundle\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.841233 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.841152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-audit-log\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.855839 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.855807 2568 generic.go:356] "Generic (PLEG): container finished" podID="23521425-31dd-4d23-90ce-e855a6b5549b" containerID="03e71056ecbb9bb61e7477154f8fa5ae61280531b904faab76fde9ce42ff5cdd" exitCode=0 Jul 02 23:02:48.856171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.855897 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghk6j" event={"ID":"23521425-31dd-4d23-90ce-e855a6b5549b","Type":"ContainerDied","Data":"03e71056ecbb9bb61e7477154f8fa5ae61280531b904faab76fde9ce42ff5cdd"} Jul 02 23:02:48.857788 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.857763 2568 generic.go:356] "Generic (PLEG): container finished" podID="89b1a49b-041c-4788-84b1-71104153a384" containerID="c0158846b17927800df8272df3071410bdd721f0f46b6d0d71645dc88d42f96e" exitCode=0 Jul 02 23:02:48.857884 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.857862 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerDied","Data":"c0158846b17927800df8272df3071410bdd721f0f46b6d0d71645dc88d42f96e"} Jul 02 23:02:48.860780 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.860698 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" event={"ID":"095e97d7-a7b1-437e-a1e2-dadc8524eaf9","Type":"ContainerStarted","Data":"176aa700f38e9b2c9e99a83bd65807cc3312b80be813050ed21ad3c45ad288e5"} Jul 02 23:02:48.860780 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.860758 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" event={"ID":"095e97d7-a7b1-437e-a1e2-dadc8524eaf9","Type":"ContainerStarted","Data":"bda1186fe8a7ebe1a3341205a92943c5899a90efce9bf98383e02ec05b9f4080"} Jul 02 23:02:48.860900 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.860789 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" event={"ID":"095e97d7-a7b1-437e-a1e2-dadc8524eaf9","Type":"ContainerStarted","Data":"053297e898194738cb2121ea396559cc81a7bc569fa9ab630301767ab38b715f"} Jul 02 23:02:48.863365 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.863340 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" event={"ID":"7583ca97-2e44-462c-b17c-cc79dc6f8589","Type":"ContainerStarted","Data":"2f29996de2fdf237eb1b33a62b47d9b5f2da3d6cc6e6a7c18df5aa0e06ed30d0"} Jul 02 23:02:48.864766 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.864736 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-rjnfd" event={"ID":"cfbde320-130d-4761-81df-3154c7c0f7e2","Type":"ContainerStarted","Data":"dd16617929c3e195b333a480e142627533471059943f713de9635d8856fe5244"} Jul 02 23:02:48.890617 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.890536 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hpdzb" podStartSLOduration=2.892713436 podStartE2EDuration="4.89052194s" totalImagesPullingTime="1.997808504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:02:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:48.889786069 +0000 UTC m=+75.963038432" watchObservedRunningTime="2026-07-02 23:02:48.89052194 +0000 UTC m=+75.963774302" Jul 02 23:02:48.914435 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.914369 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4lmt2" podStartSLOduration=2.806103959 podStartE2EDuration="4.914350158s" totalImagesPullingTime="2.108246199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:02:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:48.913029453 +0000 UTC m=+75.986281820" watchObservedRunningTime="2026-07-02 23:02:48.914350158 +0000 UTC m=+75.987602525" Jul 02 23:02:48.942285 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.942258 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-secret-metrics-server-tls\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.942368 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.942327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-client-ca-bundle\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.942480 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.942464 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-audit-log\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.942688 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.942668 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdfpq\" (UniqueName: \"kubernetes.io/projected/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-kube-api-access-rdfpq\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.943124 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.942761 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.943124 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.942825 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-metrics-server-audit-profiles\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.943124 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.942940 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-secret-metrics-server-client-certs\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.945295 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.945246 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-audit-log\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.945581 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.945540 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.946482 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.946449 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-metrics-server-audit-profiles\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.947852 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.947813 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-secret-metrics-server-client-certs\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.948405 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.948360 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-secret-metrics-server-tls\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.948902 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.948880 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-client-ca-bundle\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:48.959211 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:48.959186 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdfpq\" (UniqueName: \"kubernetes.io/projected/0d829ca4-3ab1-4c14-a42f-d21a4dd13096-kube-api-access-rdfpq\") pod \"metrics-server-67dcb868c9-w2hpl\" (UID: \"0d829ca4-3ab1-4c14-a42f-d21a4dd13096\") " pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:49.091396 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:49.091325 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:02:49.243112 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:49.242891 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:49.243112 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:49.243077 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67dcb868c9-w2hpl"] Jul 02 23:02:49.245849 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:02:49.245822 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d829ca4_3ab1_4c14_a42f_d21a4dd13096.slice/crio-59f4ac1be60dccf82631aaadd71cb7fadc0042856f42b0195a94fdda3885e09b WatchSource:0}: Error finding container 59f4ac1be60dccf82631aaadd71cb7fadc0042856f42b0195a94fdda3885e09b: Status 404 returned error can't find the container with id 59f4ac1be60dccf82631aaadd71cb7fadc0042856f42b0195a94fdda3885e09b Jul 02 23:02:49.355742 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:49.355661 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:49.475513 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:49.475446 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:49.748029 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:49.747991 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:49.873622 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:49.873590 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghk6j" event={"ID":"23521425-31dd-4d23-90ce-e855a6b5549b","Type":"ContainerStarted","Data":"1c597c2e1f0827ae51d9c32063a55db7db8f543f5da0ff6c053e34c872a63bdc"} Jul 02 23:02:49.874026 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:49.873651 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghk6j" event={"ID":"23521425-31dd-4d23-90ce-e855a6b5549b","Type":"ContainerStarted","Data":"2f14fc4430d4b9b62f543c408badf7066cb1967f4ec116d8bd6ea3fdbf329cbb"} Jul 02 23:02:49.875597 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:49.875513 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" event={"ID":"0d829ca4-3ab1-4c14-a42f-d21a4dd13096","Type":"ContainerStarted","Data":"59f4ac1be60dccf82631aaadd71cb7fadc0042856f42b0195a94fdda3885e09b"} Jul 02 23:02:49.894603 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:49.893765 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ghk6j" podStartSLOduration=4.202904875 podStartE2EDuration="5.893746794s" totalImagesPullingTime="1.59674702s" totalInitContainerRuntime="94.094899ms" isStatefulPod=true podCreationTimestamp="2026-07-02 23:02:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:49.892491167 +0000 UTC m=+76.965743531" watchObservedRunningTime="2026-07-02 23:02:49.893746794 +0000 UTC m=+76.966999158" Jul 02 23:02:50.724028 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:50.723978 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:51.886983 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:51.886947 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" event={"ID":"0d829ca4-3ab1-4c14-a42f-d21a4dd13096","Type":"ContainerStarted","Data":"b0dc77dd2cb839ebdc4486ce12f6e797cbfc1a8533ab9897b05e7f9ac14e5277"} Jul 02 23:02:51.890672 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:51.890641 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerStarted","Data":"5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a"} Jul 02 23:02:51.890782 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:51.890702 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerStarted","Data":"7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb"} Jul 02 23:02:51.890782 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:51.890728 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerStarted","Data":"21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6"} Jul 02 23:02:51.890782 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:51.890757 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerStarted","Data":"877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39"} Jul 02 23:02:51.890950 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:51.890788 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerStarted","Data":"e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954"} Jul 02 23:02:51.906729 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:51.906682 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" podStartSLOduration=1.87951424 podStartE2EDuration="3.906662464s" totalImagesPullingTime="2.027148224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:51.903509866 +0000 UTC m=+78.976762232" watchObservedRunningTime="2026-07-02 23:02:51.906662464 +0000 UTC m=+78.979914828" Jul 02 23:02:52.143390 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:52.143270 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:52.206356 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:52.206308 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:52.707445 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:52.707406 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:02:52.899308 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:52.899271 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerStarted","Data":"1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4"} Jul 02 23:02:52.924668 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:52.924615 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.801854556 podStartE2EDuration="7.92455773s" totalImagesPullingTime="5.102930633s" totalInitContainerRuntime="19.772541ms" isStatefulPod=false podCreationTimestamp="2026-07-02 23:02:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:02:52.922937613 +0000 UTC m=+79.996189977" watchObservedRunningTime="2026-07-02 23:02:52.92455773 +0000 UTC m=+79.997810093" Jul 02 23:02:53.161949 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:53.161864 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:53.261208 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:53.261168 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:53.646787 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:53.646749 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:53.754031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:53.753996 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:54.777060 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:54.777028 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-55fbb5d77-ff26t" Jul 02 23:02:54.904112 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:54.904083 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:55.006954 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:55.006915 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:55.644878 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:55.644849 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:55.743730 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:55.743699 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:55.812974 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:55.812939 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:02:57.721524 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:02:57.721479 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" podUID="72042bb7-03a7-4133-b397-ecb5c0052cea" containerName="registry" containerID="cri-o://0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5" gracePeriod=30 Jul 02 23:03:02.540474 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:02.540441 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:02.641540 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:02.641501 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:02.703510 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:02.703477 2568 patch_prober.go:28] interesting pod/image-registry-84f6dfc7cc-pjfxq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.133.0.6:5000/healthz\": dial tcp 10.133.0.6:5000: connect: connection refused" start-of-body= Jul 02 23:03:02.703675 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:02.703519 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" podUID="72042bb7-03a7-4133-b397-ecb5c0052cea" containerName="registry" probeResult="failure" output="Get \"https://10.133.0.6:5000/healthz\": dial tcp 10.133.0.6:5000: connect: connection refused" Jul 02 23:03:04.671353 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.671326 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:03:04.776649 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.776587 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm4h2\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-kube-api-access-jm4h2\") pod \"72042bb7-03a7-4133-b397-ecb5c0052cea\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " Jul 02 23:03:04.776773 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.776654 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls\") pod \"72042bb7-03a7-4133-b397-ecb5c0052cea\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " Jul 02 23:03:04.776773 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.776682 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-installation-pull-secrets\") pod \"72042bb7-03a7-4133-b397-ecb5c0052cea\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " Jul 02 23:03:04.776773 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.776707 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-trusted-ca\") pod \"72042bb7-03a7-4133-b397-ecb5c0052cea\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " Jul 02 23:03:04.776773 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.776727 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-bound-sa-token\") pod \"72042bb7-03a7-4133-b397-ecb5c0052cea\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " Jul 02 23:03:04.776773 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.776748 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-image-registry-private-configuration\") pod \"72042bb7-03a7-4133-b397-ecb5c0052cea\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " Jul 02 23:03:04.776773 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.776769 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-certificates\") pod \"72042bb7-03a7-4133-b397-ecb5c0052cea\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " Jul 02 23:03:04.777073 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.776822 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72042bb7-03a7-4133-b397-ecb5c0052cea-ca-trust-extracted\") pod \"72042bb7-03a7-4133-b397-ecb5c0052cea\" (UID: \"72042bb7-03a7-4133-b397-ecb5c0052cea\") " Jul 02 23:03:04.777326 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.777233 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "72042bb7-03a7-4133-b397-ecb5c0052cea" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:03:04.777518 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.777491 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "72042bb7-03a7-4133-b397-ecb5c0052cea" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:03:04.779466 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.779435 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "72042bb7-03a7-4133-b397-ecb5c0052cea" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:03:04.779615 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.779487 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "72042bb7-03a7-4133-b397-ecb5c0052cea" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:03:04.779615 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.779528 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "72042bb7-03a7-4133-b397-ecb5c0052cea" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:03:04.779722 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.779622 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "72042bb7-03a7-4133-b397-ecb5c0052cea" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:03:04.779722 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.779673 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-kube-api-access-jm4h2" (OuterVolumeSpecName: "kube-api-access-jm4h2") pod "72042bb7-03a7-4133-b397-ecb5c0052cea" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea"). InnerVolumeSpecName "kube-api-access-jm4h2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:03:04.785506 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.785484 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72042bb7-03a7-4133-b397-ecb5c0052cea-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "72042bb7-03a7-4133-b397-ecb5c0052cea" (UID: "72042bb7-03a7-4133-b397-ecb5c0052cea"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:03:04.878125 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.878102 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm4h2\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-kube-api-access-jm4h2\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:03:04.878210 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.878127 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-tls\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:03:04.878210 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.878142 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-installation-pull-secrets\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:03:04.878210 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.878152 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-trusted-ca\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:03:04.878210 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.878160 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72042bb7-03a7-4133-b397-ecb5c0052cea-bound-sa-token\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:03:04.878210 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.878169 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72042bb7-03a7-4133-b397-ecb5c0052cea-image-registry-private-configuration\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:03:04.878210 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.878178 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72042bb7-03a7-4133-b397-ecb5c0052cea-registry-certificates\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:03:04.878210 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.878186 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72042bb7-03a7-4133-b397-ecb5c0052cea-ca-trust-extracted\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:03:04.947529 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.947506 2568 generic.go:356] "Generic (PLEG): container finished" podID="72042bb7-03a7-4133-b397-ecb5c0052cea" containerID="0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5" exitCode=0 Jul 02 23:03:04.947648 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.947559 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" Jul 02 23:03:04.947648 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.947596 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" event={"ID":"72042bb7-03a7-4133-b397-ecb5c0052cea","Type":"ContainerDied","Data":"0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5"} Jul 02 23:03:04.947648 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.947646 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f6dfc7cc-pjfxq" event={"ID":"72042bb7-03a7-4133-b397-ecb5c0052cea","Type":"ContainerDied","Data":"4284f4b8e3e8f9e8f087ab201bbb53ba3dd9a920fa48965d24d3271cb2bac960"} Jul 02 23:03:04.947798 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.947673 2568 scope.go:117] "RemoveContainer" containerID="0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5" Jul 02 23:03:04.958017 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.957996 2568 scope.go:117] "RemoveContainer" containerID="0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5" Jul 02 23:03:04.958316 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:03:04.958298 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5\": container with ID starting with 0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5 not found: ID does not exist" containerID="0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5" Jul 02 23:03:04.958406 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.958323 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5"} err="failed to get container status \"0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5\": rpc error: code = NotFound desc = could not find container \"0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5\": container with ID starting with 0c1358002517d2ad8aa6d571b33928e274dc52506d2b917aa95e41d4de0db9e5 not found: ID does not exist" Jul 02 23:03:04.970130 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.970104 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84f6dfc7cc-pjfxq"] Jul 02 23:03:04.973348 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:04.973326 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84f6dfc7cc-pjfxq"] Jul 02 23:03:05.468242 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:05.468202 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72042bb7-03a7-4133-b397-ecb5c0052cea" path="/var/lib/kubelet/pods/72042bb7-03a7-4133-b397-ecb5c0052cea/volumes" Jul 02 23:03:05.960641 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:05.960600 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-rjnfd" event={"ID":"cfbde320-130d-4761-81df-3154c7c0f7e2","Type":"ContainerStarted","Data":"cf299c04dd77b097603854d5656ed14e64ca6b498902c7cd3446dbe483044b05"} Jul 02 23:03:05.961226 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:05.960842 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-rjnfd" Jul 02 23:03:05.975036 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:05.975013 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-rjnfd" Jul 02 23:03:05.979981 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:05.979928 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-rjnfd" podStartSLOduration=3.044681626 podStartE2EDuration="19.979911982s" totalImagesPullingTime="16.935230356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:03:05.977482845 +0000 UTC m=+93.050735209" watchObservedRunningTime="2026-07-02 23:03:05.979911982 +0000 UTC m=+93.053164345" Jul 02 23:03:06.112030 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:06.111977 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:06.168695 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:06.168654 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:07.343357 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:07.343316 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:07.406329 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:07.406291 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:08.156444 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:08.156411 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:08.214144 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:08.214112 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:09.091992 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:09.091965 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:03:09.092502 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:09.092018 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:03:16.388449 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:16.388413 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:16.848961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:16.848887 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hzb5r" Jul 02 23:03:17.339064 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:17.339037 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:21.941624 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:21.941588 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:22.042310 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:22.042278 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:22.297335 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:22.297271 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:22.353344 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:22.353317 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:23.156692 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:23.156647 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:23.210894 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:23.210868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:29.096738 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:29.096700 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:03:29.100397 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:29.100373 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-67dcb868c9-w2hpl" Jul 02 23:03:30.805720 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:30.805690 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:30.866996 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:30.866968 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:35.509997 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:35.509964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:38.156900 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:38.156863 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:03:38.214323 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:03:38.214295 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:03.722733 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:03.722699 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:03.846838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:03.846803 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:04.778503 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:04.778470 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:04:04.778923 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:04.778799 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="alertmanager" containerID="cri-o://e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954" gracePeriod=120 Jul 02 23:04:04.778923 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:04.778823 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy" containerID="cri-o://7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb" gracePeriod=120 Jul 02 23:04:04.778923 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:04.778857 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy-web" containerID="cri-o://21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6" gracePeriod=120 Jul 02 23:04:04.778923 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:04.778871 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="config-reloader" containerID="cri-o://877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39" gracePeriod=120 Jul 02 23:04:04.778923 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:04.778906 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy-metric" containerID="cri-o://5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a" gracePeriod=120 Jul 02 23:04:04.779168 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:04.778851 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="prom-label-proxy" containerID="cri-o://1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4" gracePeriod=120 Jul 02 23:04:05.182411 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:05.182380 2568 generic.go:356] "Generic (PLEG): container finished" podID="89b1a49b-041c-4788-84b1-71104153a384" containerID="1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4" exitCode=0 Jul 02 23:04:05.182411 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:05.182405 2568 generic.go:356] "Generic (PLEG): container finished" podID="89b1a49b-041c-4788-84b1-71104153a384" containerID="7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb" exitCode=0 Jul 02 23:04:05.182411 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:05.182413 2568 generic.go:356] "Generic (PLEG): container finished" podID="89b1a49b-041c-4788-84b1-71104153a384" containerID="21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6" exitCode=0 Jul 02 23:04:05.182411 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:05.182419 2568 generic.go:356] "Generic (PLEG): container finished" podID="89b1a49b-041c-4788-84b1-71104153a384" containerID="877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39" exitCode=0 Jul 02 23:04:05.182706 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:05.182426 2568 generic.go:356] "Generic (PLEG): container finished" podID="89b1a49b-041c-4788-84b1-71104153a384" containerID="e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954" exitCode=0 Jul 02 23:04:05.182706 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:05.182448 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerDied","Data":"1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4"} Jul 02 23:04:05.182706 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:05.182518 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerDied","Data":"7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb"} Jul 02 23:04:05.182706 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:05.182545 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerDied","Data":"21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6"} Jul 02 23:04:05.182706 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:05.182597 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerDied","Data":"877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39"} Jul 02 23:04:05.182706 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:05.182623 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerDied","Data":"e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954"} Jul 02 23:04:06.011466 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.011445 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.143723 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.143690 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-cluster-tls-config\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.143883 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.143729 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-main-db\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.143883 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.143767 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.143883 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.143797 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-trusted-ca-bundle\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.143883 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.143826 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-metric\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.143883 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.143860 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-web\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.144150 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.143898 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-tls-assets\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.144150 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.143928 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-main-tls\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.144150 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.143959 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-web-config\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.144150 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.143986 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-config-out\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.144150 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.144015 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-metrics-client-ca\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.144150 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.144058 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-config-volume\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.144150 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.144089 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:04:06.144150 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.144095 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdmqw\" (UniqueName: \"kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-kube-api-access-bdmqw\") pod \"89b1a49b-041c-4788-84b1-71104153a384\" (UID: \"89b1a49b-041c-4788-84b1-71104153a384\") " Jul 02 23:04:06.144545 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.144229 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:04:06.144545 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.144375 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.144545 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.144401 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-alertmanager-main-db\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.146703 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.146674 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:04:06.147669 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.147162 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:04:06.147669 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.147200 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:04:06.147669 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.147282 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:04:06.147669 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.147438 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:04:06.147669 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.147521 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-kube-api-access-bdmqw" (OuterVolumeSpecName: "kube-api-access-bdmqw") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "kube-api-access-bdmqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:04:06.147669 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.147523 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:04:06.148353 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.148336 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-config-out" (OuterVolumeSpecName: "config-out") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:04:06.148602 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.148544 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-config-volume" (OuterVolumeSpecName: "config-volume") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:04:06.151239 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.151217 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:04:06.157501 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.157478 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-web-config" (OuterVolumeSpecName: "web-config") pod "89b1a49b-041c-4788-84b1-71104153a384" (UID: "89b1a49b-041c-4788-84b1-71104153a384"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:04:06.189031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.189000 2568 generic.go:356] "Generic (PLEG): container finished" podID="89b1a49b-041c-4788-84b1-71104153a384" containerID="5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a" exitCode=0 Jul 02 23:04:06.189132 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.189067 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerDied","Data":"5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a"} Jul 02 23:04:06.189187 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.189072 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.189187 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.189160 2568 scope.go:117] "RemoveContainer" containerID="1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4" Jul 02 23:04:06.189300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.189144 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"89b1a49b-041c-4788-84b1-71104153a384","Type":"ContainerDied","Data":"22976886022cc520a9ee7648dc0e562452b8b0e4f7701cdb51b5363a0b6137b1"} Jul 02 23:04:06.197277 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.197258 2568 scope.go:117] "RemoveContainer" containerID="5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a" Jul 02 23:04:06.203789 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.203770 2568 scope.go:117] "RemoveContainer" containerID="7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb" Jul 02 23:04:06.209935 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.209918 2568 scope.go:117] "RemoveContainer" containerID="21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6" Jul 02 23:04:06.212866 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.212846 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:04:06.216451 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.216421 2568 scope.go:117] "RemoveContainer" containerID="877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39" Jul 02 23:04:06.219282 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.219261 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:04:06.223343 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.223327 2568 scope.go:117] "RemoveContainer" containerID="e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954" Jul 02 23:04:06.229713 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.229698 2568 scope.go:117] "RemoveContainer" containerID="c0158846b17927800df8272df3071410bdd721f0f46b6d0d71645dc88d42f96e" Jul 02 23:04:06.235980 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.235961 2568 scope.go:117] "RemoveContainer" containerID="1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4" Jul 02 23:04:06.236256 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:04:06.236234 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4\": container with ID starting with 1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4 not found: ID does not exist" containerID="1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4" Jul 02 23:04:06.236313 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.236274 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4"} err="failed to get container status \"1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4\": rpc error: code = NotFound desc = could not find container \"1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4\": container with ID starting with 1793bbcec99dff34a8c298fdb3753aff93234762ad6391aae11796895dda52b4 not found: ID does not exist" Jul 02 23:04:06.236313 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.236293 2568 scope.go:117] "RemoveContainer" containerID="5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a" Jul 02 23:04:06.236558 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:04:06.236540 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a\": container with ID starting with 5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a not found: ID does not exist" containerID="5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a" Jul 02 23:04:06.236739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.236589 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a"} err="failed to get container status \"5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a\": rpc error: code = NotFound desc = could not find container \"5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a\": container with ID starting with 5464972600fa84f4f57d766984192c159b5716920b205b0a74e2fdbbb736a93a not found: ID does not exist" Jul 02 23:04:06.236863 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.236742 2568 scope.go:117] "RemoveContainer" containerID="7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb" Jul 02 23:04:06.237052 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:04:06.237031 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb\": container with ID starting with 7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb not found: ID does not exist" containerID="7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb" Jul 02 23:04:06.237116 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.237056 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb"} err="failed to get container status \"7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb\": rpc error: code = NotFound desc = could not find container \"7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb\": container with ID starting with 7055064e68958b1a931d66b1a58cc18c2dd737ff7e5b2e866eaafc6f46e10bdb not found: ID does not exist" Jul 02 23:04:06.237116 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.237076 2568 scope.go:117] "RemoveContainer" containerID="21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6" Jul 02 23:04:06.237349 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:04:06.237327 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6\": container with ID starting with 21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6 not found: ID does not exist" containerID="21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6" Jul 02 23:04:06.237390 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.237357 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6"} err="failed to get container status \"21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6\": rpc error: code = NotFound desc = could not find container \"21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6\": container with ID starting with 21784abb06a6c6e4aacdee6d143be771b5c1ce54b0d24b0e08b3414be1c87af6 not found: ID does not exist" Jul 02 23:04:06.237390 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.237373 2568 scope.go:117] "RemoveContainer" containerID="877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39" Jul 02 23:04:06.237613 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:04:06.237597 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39\": container with ID starting with 877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39 not found: ID does not exist" containerID="877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39" Jul 02 23:04:06.237660 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.237619 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39"} err="failed to get container status \"877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39\": rpc error: code = NotFound desc = could not find container \"877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39\": container with ID starting with 877f139dcf2e61ec98b8655e05311cfc7ada716c6ba951f60902fcc33a07fa39 not found: ID does not exist" Jul 02 23:04:06.237660 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.237633 2568 scope.go:117] "RemoveContainer" containerID="e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954" Jul 02 23:04:06.237843 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:04:06.237827 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954\": container with ID starting with e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954 not found: ID does not exist" containerID="e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954" Jul 02 23:04:06.237884 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.237848 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954"} err="failed to get container status \"e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954\": rpc error: code = NotFound desc = could not find container \"e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954\": container with ID starting with e260ee07b942a352f7113405a37a2350ed13882e128cc96f8771a2410879d954 not found: ID does not exist" Jul 02 23:04:06.237884 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.237864 2568 scope.go:117] "RemoveContainer" containerID="c0158846b17927800df8272df3071410bdd721f0f46b6d0d71645dc88d42f96e" Jul 02 23:04:06.238077 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:04:06.238060 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0158846b17927800df8272df3071410bdd721f0f46b6d0d71645dc88d42f96e\": container with ID starting with c0158846b17927800df8272df3071410bdd721f0f46b6d0d71645dc88d42f96e not found: ID does not exist" containerID="c0158846b17927800df8272df3071410bdd721f0f46b6d0d71645dc88d42f96e" Jul 02 23:04:06.238126 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.238084 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0158846b17927800df8272df3071410bdd721f0f46b6d0d71645dc88d42f96e"} err="failed to get container status \"c0158846b17927800df8272df3071410bdd721f0f46b6d0d71645dc88d42f96e\": rpc error: code = NotFound desc = could not find container \"c0158846b17927800df8272df3071410bdd721f0f46b6d0d71645dc88d42f96e\": container with ID starting with c0158846b17927800df8272df3071410bdd721f0f46b6d0d71645dc88d42f96e not found: ID does not exist" Jul 02 23:04:06.244930 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.244906 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-tls-assets\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.245012 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.244934 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-main-tls\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.245012 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.244970 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-web-config\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.245012 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.244985 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/89b1a49b-041c-4788-84b1-71104153a384-config-out\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.245012 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.244998 2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89b1a49b-041c-4788-84b1-71104153a384-metrics-client-ca\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.245012 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.245010 2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-config-volume\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.245268 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.245022 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdmqw\" (UniqueName: \"kubernetes.io/projected/89b1a49b-041c-4788-84b1-71104153a384-kube-api-access-bdmqw\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.245268 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.245035 2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-cluster-tls-config\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.245268 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.245049 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.245268 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.245062 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.245268 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.245075 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89b1a49b-041c-4788-84b1-71104153a384-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:04:06.246007 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.245987 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:04:06.246257 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246242 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="prom-label-proxy" Jul 02 23:04:06.246324 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246260 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="prom-label-proxy" Jul 02 23:04:06.246324 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246277 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="init-config-reloader" Jul 02 23:04:06.246324 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246286 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="init-config-reloader" Jul 02 23:04:06.246324 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246299 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="config-reloader" Jul 02 23:04:06.246324 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246308 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="config-reloader" Jul 02 23:04:06.246324 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246319 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy-metric" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246328 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy-metric" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246339 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72042bb7-03a7-4133-b397-ecb5c0052cea" containerName="registry" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246348 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="72042bb7-03a7-4133-b397-ecb5c0052cea" containerName="registry" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246359 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="alertmanager" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246368 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="alertmanager" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246379 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy-web" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246389 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy-web" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246402 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246442 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246510 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="72042bb7-03a7-4133-b397-ecb5c0052cea" containerName="registry" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246522 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="prom-label-proxy" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246532 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="alertmanager" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246542 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246550 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy-metric" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246558 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="kube-rbac-proxy-web" Jul 02 23:04:06.246638 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.246584 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="89b1a49b-041c-4788-84b1-71104153a384" containerName="config-reloader" Jul 02 23:04:06.249757 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.249740 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.252529 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.252480 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 23:04:06.252529 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.252492 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 23:04:06.252700 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.252548 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 23:04:06.252700 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.252590 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 23:04:06.252700 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.252671 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 23:04:06.252936 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.252921 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 23:04:06.253024 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.252938 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 23:04:06.253024 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.252973 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 23:04:06.253100 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.252993 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wz6tb\"" Jul 02 23:04:06.262690 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.262667 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 23:04:06.266815 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.266795 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:04:06.346128 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346099 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346128 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346130 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346307 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346147 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/67ad99e8-4256-47fc-a2fe-72ae15865c51-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346307 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346173 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sdrp\" (UniqueName: \"kubernetes.io/projected/67ad99e8-4256-47fc-a2fe-72ae15865c51-kube-api-access-7sdrp\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346307 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346199 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ad99e8-4256-47fc-a2fe-72ae15865c51-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346307 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346236 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346307 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346258 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346307 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346299 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67ad99e8-4256-47fc-a2fe-72ae15865c51-config-out\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346531 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346314 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-web-config\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346531 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346531 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346352 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67ad99e8-4256-47fc-a2fe-72ae15865c51-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346531 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346399 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67ad99e8-4256-47fc-a2fe-72ae15865c51-tls-assets\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.346531 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.346426 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-config-volume\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447006 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.446911 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sdrp\" (UniqueName: \"kubernetes.io/projected/67ad99e8-4256-47fc-a2fe-72ae15865c51-kube-api-access-7sdrp\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447006 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.446954 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ad99e8-4256-47fc-a2fe-72ae15865c51-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447006 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.446988 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447006 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.447006 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447318 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.447025 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67ad99e8-4256-47fc-a2fe-72ae15865c51-config-out\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447318 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.447040 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-web-config\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447318 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.447058 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447318 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.447075 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67ad99e8-4256-47fc-a2fe-72ae15865c51-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447318 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.447101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67ad99e8-4256-47fc-a2fe-72ae15865c51-tls-assets\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447318 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.447201 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-config-volume\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447318 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.447281 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447680 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.447324 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447680 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.447350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/67ad99e8-4256-47fc-a2fe-72ae15865c51-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.447778 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.447749 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/67ad99e8-4256-47fc-a2fe-72ae15865c51-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.448300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.448271 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67ad99e8-4256-47fc-a2fe-72ae15865c51-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.448647 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.448620 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ad99e8-4256-47fc-a2fe-72ae15865c51-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.450276 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.450153 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67ad99e8-4256-47fc-a2fe-72ae15865c51-tls-assets\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.450370 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.450339 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.450438 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.450408 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.450583 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.450545 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67ad99e8-4256-47fc-a2fe-72ae15865c51-config-out\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.450793 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.450775 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.450793 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.450789 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-web-config\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.450876 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.450801 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.451166 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.451151 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.451760 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.451735 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/67ad99e8-4256-47fc-a2fe-72ae15865c51-config-volume\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.455354 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.455334 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sdrp\" (UniqueName: \"kubernetes.io/projected/67ad99e8-4256-47fc-a2fe-72ae15865c51-kube-api-access-7sdrp\") pod \"alertmanager-main-0\" (UID: \"67ad99e8-4256-47fc-a2fe-72ae15865c51\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.563502 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.563461 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:04:06.690158 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:06.690115 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:04:06.693615 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:04:06.693587 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ad99e8_4256_47fc_a2fe_72ae15865c51.slice/crio-ad21a069cba39675e5c5205a457fe491c3667cfe50bce354bd4545f4ac39d68d WatchSource:0}: Error finding container ad21a069cba39675e5c5205a457fe491c3667cfe50bce354bd4545f4ac39d68d: Status 404 returned error can't find the container with id ad21a069cba39675e5c5205a457fe491c3667cfe50bce354bd4545f4ac39d68d Jul 02 23:04:07.195551 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:07.195513 2568 generic.go:356] "Generic (PLEG): container finished" podID="67ad99e8-4256-47fc-a2fe-72ae15865c51" containerID="0dd87bb9336f4b60fd0375ed7e06ecda924c6d8e8785a4cc7238de31fc92ea24" exitCode=0 Jul 02 23:04:07.195980 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:07.195611 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67ad99e8-4256-47fc-a2fe-72ae15865c51","Type":"ContainerDied","Data":"0dd87bb9336f4b60fd0375ed7e06ecda924c6d8e8785a4cc7238de31fc92ea24"} Jul 02 23:04:07.195980 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:07.195678 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67ad99e8-4256-47fc-a2fe-72ae15865c51","Type":"ContainerStarted","Data":"ad21a069cba39675e5c5205a457fe491c3667cfe50bce354bd4545f4ac39d68d"} Jul 02 23:04:07.469411 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:07.469381 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89b1a49b-041c-4788-84b1-71104153a384" path="/var/lib/kubelet/pods/89b1a49b-041c-4788-84b1-71104153a384/volumes" Jul 02 23:04:08.202017 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.201985 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67ad99e8-4256-47fc-a2fe-72ae15865c51","Type":"ContainerStarted","Data":"b879a5760dbd16a100aef735b4e592276e7ff00119d79e817dc0331108f59024"} Jul 02 23:04:08.202390 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.202052 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67ad99e8-4256-47fc-a2fe-72ae15865c51","Type":"ContainerStarted","Data":"8255fe84ff2dd6a81d9dda2977c418fbc4b1013a8b3cc78f81720cd9f7a11961"} Jul 02 23:04:08.202390 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.202077 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67ad99e8-4256-47fc-a2fe-72ae15865c51","Type":"ContainerStarted","Data":"ecb650c8c24fe4e6c0e8b641f85602dc5a1747580b4fd7110fdc2988a5459a2c"} Jul 02 23:04:08.202390 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.202105 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67ad99e8-4256-47fc-a2fe-72ae15865c51","Type":"ContainerStarted","Data":"73ae5a2825b933011f5d99d04404666fb5c6cdf5da48533ccf2fd8cce3cc6598"} Jul 02 23:04:08.202390 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.202130 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67ad99e8-4256-47fc-a2fe-72ae15865c51","Type":"ContainerStarted","Data":"5dd2090b50b4741b8cec071159e304dc1bb40a256838271416273452d9179f06"} Jul 02 23:04:08.202390 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.202155 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67ad99e8-4256-47fc-a2fe-72ae15865c51","Type":"ContainerStarted","Data":"0255f74bc3e9621f824501b554cb6837d17d27679b1579e2fc6437fdd0c51f6d"} Jul 02 23:04:08.233377 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.233320 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.217460756 podStartE2EDuration="2.233300211s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.839455ms" isStatefulPod=false podCreationTimestamp="2026-07-02 23:04:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:04:08.230458433 +0000 UTC m=+155.303710800" watchObservedRunningTime="2026-07-02 23:04:08.233300211 +0000 UTC m=+155.306552574" Jul 02 23:04:08.795826 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.795795 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-559fd44f86-v8nw5"] Jul 02 23:04:08.798147 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.798124 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:08.800630 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.800608 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 02 23:04:08.800780 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.800632 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 02 23:04:08.801020 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.801003 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 02 23:04:08.801095 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.801008 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 02 23:04:08.801095 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.801067 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 02 23:04:08.801320 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.801306 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rw64p\"" Jul 02 23:04:08.805698 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.805584 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 02 23:04:08.811610 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.811588 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-559fd44f86-v8nw5"] Jul 02 23:04:08.968986 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.968950 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-serving-certs-ca-bundle\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:08.968986 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.968985 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-federate-client-tls\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:08.969175 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.969026 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-telemeter-trusted-ca-bundle\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:08.969175 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.969070 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-telemeter-client-tls\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:08.969175 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.969089 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:08.969175 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.969120 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-metrics-client-ca\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:08.969175 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.969138 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd2jg\" (UniqueName: \"kubernetes.io/projected/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-kube-api-access-xd2jg\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:08.969360 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:08.969207 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-secret-telemeter-client\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.070282 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.070200 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-metrics-client-ca\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.070282 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.070246 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd2jg\" (UniqueName: \"kubernetes.io/projected/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-kube-api-access-xd2jg\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.070473 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.070299 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-secret-telemeter-client\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.070473 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.070323 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-serving-certs-ca-bundle\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.070473 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.070346 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-federate-client-tls\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.070473 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.070389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-telemeter-trusted-ca-bundle\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.070473 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.070419 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-telemeter-client-tls\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.070473 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.070446 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.071113 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.071078 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-serving-certs-ca-bundle\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.071113 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.071105 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-metrics-client-ca\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.071477 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.071449 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-telemeter-trusted-ca-bundle\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.072879 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.072854 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-federate-client-tls\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.072996 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.072979 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-secret-telemeter-client\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.073193 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.073174 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.073263 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.073230 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-telemeter-client-tls\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.078465 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.078444 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd2jg\" (UniqueName: \"kubernetes.io/projected/a6f84056-2ca0-4eb4-9125-85cfde48a8ec-kube-api-access-xd2jg\") pod \"telemeter-client-559fd44f86-v8nw5\" (UID: \"a6f84056-2ca0-4eb4-9125-85cfde48a8ec\") " pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.108680 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.108656 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" Jul 02 23:04:09.237040 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:09.237020 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-559fd44f86-v8nw5"] Jul 02 23:04:09.238606 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:04:09.238580 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6f84056_2ca0_4eb4_9125_85cfde48a8ec.slice/crio-b572ead43fcce78eae8baa8d51fcc32e8835d6a2590afaa98768ba1cdf4271a0 WatchSource:0}: Error finding container b572ead43fcce78eae8baa8d51fcc32e8835d6a2590afaa98768ba1cdf4271a0: Status 404 returned error can't find the container with id b572ead43fcce78eae8baa8d51fcc32e8835d6a2590afaa98768ba1cdf4271a0 Jul 02 23:04:10.211527 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:10.211491 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" event={"ID":"a6f84056-2ca0-4eb4-9125-85cfde48a8ec","Type":"ContainerStarted","Data":"b572ead43fcce78eae8baa8d51fcc32e8835d6a2590afaa98768ba1cdf4271a0"} Jul 02 23:04:11.217584 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:11.217534 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" event={"ID":"a6f84056-2ca0-4eb4-9125-85cfde48a8ec","Type":"ContainerStarted","Data":"49ee82be43e4593f40ce9cc20c3d1e5cdf187adda0af0b6f45f39e2c7904df6e"} Jul 02 23:04:12.223352 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:12.223317 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" event={"ID":"a6f84056-2ca0-4eb4-9125-85cfde48a8ec","Type":"ContainerStarted","Data":"948b205574c55f411ec33e49e1115cd38b258028c334e45d5c000e589a18b471"} Jul 02 23:04:12.223762 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:12.223377 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" event={"ID":"a6f84056-2ca0-4eb4-9125-85cfde48a8ec","Type":"ContainerStarted","Data":"95b093c8696175d12d672b9b55412562cbc84fc7e5fe92ea091ca5a68d9f6e5a"} Jul 02 23:04:12.247288 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:12.247242 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-559fd44f86-v8nw5" podStartSLOduration=2.335679895 podStartE2EDuration="4.247227687s" totalImagesPullingTime="1.911547792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:04:12.245352963 +0000 UTC m=+159.318605327" watchObservedRunningTime="2026-07-02 23:04:12.247227687 +0000 UTC m=+159.320480082" Jul 02 23:04:12.965558 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:12.965522 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:13.062225 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:13.062185 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:13.127673 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:13.127643 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:13.678451 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:13.678405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:13.733518 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:13.733491 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:14.776490 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:14.776454 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:23.502553 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:23.502513 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:23.603648 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:23.603607 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:23.655105 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:23.655069 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:04:51.047677 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:04:51.047645 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:18.964596 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:18.964521 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:19.040117 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:19.040088 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:27.709285 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:27.709253 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:27.763736 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:27.763709 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:27.850970 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:27.850942 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:27.903378 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:27.903357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:28.451078 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:28.451052 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:28.505435 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:28.505412 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:30.956141 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:30.956108 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:31.014041 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:31.014014 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:38.162400 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:38.162365 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:38.262759 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:38.262727 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:38.319331 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:38.319303 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:38.781157 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:38.781123 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:38.881999 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:38.881966 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:05:38.938926 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:05:38.938897 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:10.261457 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:10.261382 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:10.357891 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:10.357863 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:10.942273 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:10.942243 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4"] Jul 02 23:06:10.945711 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:10.945693 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:10.948611 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:10.948587 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tdjmp\"" Jul 02 23:06:10.948731 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:10.948587 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 23:06:10.949600 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:10.949582 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 23:06:10.962526 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:10.962502 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4"] Jul 02 23:06:11.083999 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.083971 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z9pd\" (UniqueName: \"kubernetes.io/projected/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-kube-api-access-9z9pd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:11.084105 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.084022 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:11.084105 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.084092 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:11.184740 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.184714 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:11.184853 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.184757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z9pd\" (UniqueName: \"kubernetes.io/projected/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-kube-api-access-9z9pd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:11.184853 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.184818 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:11.185079 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.185061 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:11.185161 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.185136 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:11.192865 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.192809 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z9pd\" (UniqueName: \"kubernetes.io/projected/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-kube-api-access-9z9pd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:11.255367 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.255347 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:11.377100 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.377074 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4"] Jul 02 23:06:11.379331 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:06:11.379302 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7187a6cf_2382_4ae7_96fa_30c95a2f6c4e.slice/crio-c847790b9a5c4585eb2d725114b40eaee68815d4d26716df42b3b8af57c97cb4 WatchSource:0}: Error finding container c847790b9a5c4585eb2d725114b40eaee68815d4d26716df42b3b8af57c97cb4: Status 404 returned error can't find the container with id c847790b9a5c4585eb2d725114b40eaee68815d4d26716df42b3b8af57c97cb4 Jul 02 23:06:11.649907 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:11.649881 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" event={"ID":"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e","Type":"ContainerStarted","Data":"c847790b9a5c4585eb2d725114b40eaee68815d4d26716df42b3b8af57c97cb4"} Jul 02 23:06:17.676984 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:17.676947 2568 generic.go:356] "Generic (PLEG): container finished" podID="7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" containerID="30f1b7b635cf44b540fa1a784b9a54f3e8d8f330b1c963115f3b44a77a1e6055" exitCode=0 Jul 02 23:06:17.677271 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:17.677028 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" event={"ID":"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e","Type":"ContainerDied","Data":"30f1b7b635cf44b540fa1a784b9a54f3e8d8f330b1c963115f3b44a77a1e6055"} Jul 02 23:06:29.723351 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:29.723321 2568 generic.go:356] "Generic (PLEG): container finished" podID="7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" containerID="6a6db3e3eeb67f44a919420193ef26c7e6f0f4053f8cbaa4c1d81fb15a600a0a" exitCode=0 Jul 02 23:06:29.723663 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:29.723397 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" event={"ID":"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e","Type":"ContainerDied","Data":"6a6db3e3eeb67f44a919420193ef26c7e6f0f4053f8cbaa4c1d81fb15a600a0a"} Jul 02 23:06:33.348678 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:33.348653 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:06:33.350648 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:33.350614 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:06:33.352486 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:33.352468 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 23:06:37.759138 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:37.759101 2568 generic.go:356] "Generic (PLEG): container finished" podID="7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" containerID="d6b047a947481e11e1fc2ef05b407beb5f7bac60ddc492b7f958649c254fef39" exitCode=0 Jul 02 23:06:37.759515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:37.759175 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" event={"ID":"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e","Type":"ContainerDied","Data":"d6b047a947481e11e1fc2ef05b407beb5f7bac60ddc492b7f958649c254fef39"} Jul 02 23:06:38.887262 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:38.887240 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:38.992167 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:38.992135 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z9pd\" (UniqueName: \"kubernetes.io/projected/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-kube-api-access-9z9pd\") pod \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " Jul 02 23:06:38.992315 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:38.992186 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-util\") pod \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " Jul 02 23:06:38.992315 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:38.992233 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-bundle\") pod \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\" (UID: \"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e\") " Jul 02 23:06:38.992925 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:38.992891 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-bundle" (OuterVolumeSpecName: "bundle") pod "7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" (UID: "7187a6cf-2382-4ae7-96fa-30c95a2f6c4e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:06:38.994308 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:38.994283 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-kube-api-access-9z9pd" (OuterVolumeSpecName: "kube-api-access-9z9pd") pod "7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" (UID: "7187a6cf-2382-4ae7-96fa-30c95a2f6c4e"). InnerVolumeSpecName "kube-api-access-9z9pd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:06:38.996799 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:38.996774 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-util" (OuterVolumeSpecName: "util") pod "7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" (UID: "7187a6cf-2382-4ae7-96fa-30c95a2f6c4e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:06:39.092741 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:39.092683 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-bundle\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:06:39.092741 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:39.092704 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z9pd\" (UniqueName: \"kubernetes.io/projected/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-kube-api-access-9z9pd\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:06:39.092741 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:39.092716 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7187a6cf-2382-4ae7-96fa-30c95a2f6c4e-util\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:06:39.768685 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:39.768596 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" Jul 02 23:06:39.768685 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:39.768615 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k8vj4" event={"ID":"7187a6cf-2382-4ae7-96fa-30c95a2f6c4e","Type":"ContainerDied","Data":"c847790b9a5c4585eb2d725114b40eaee68815d4d26716df42b3b8af57c97cb4"} Jul 02 23:06:39.768685 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:39.768657 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c847790b9a5c4585eb2d725114b40eaee68815d4d26716df42b3b8af57c97cb4" Jul 02 23:06:41.796775 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:41.796731 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:43.207265 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.207228 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:43.722157 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.722121 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr"] Jul 02 23:06:43.722474 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.722460 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" containerName="util" Jul 02 23:06:43.722539 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.722476 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" containerName="util" Jul 02 23:06:43.722539 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.722493 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" containerName="extract" Jul 02 23:06:43.722539 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.722500 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" containerName="extract" Jul 02 23:06:43.722539 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.722524 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" containerName="pull" Jul 02 23:06:43.722539 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.722533 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" containerName="pull" Jul 02 23:06:43.722732 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.722600 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7187a6cf-2382-4ae7-96fa-30c95a2f6c4e" containerName="extract" Jul 02 23:06:43.725809 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.725793 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr" Jul 02 23:06:43.729276 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.729246 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:06:43.729551 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.729534 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-mvcqj\"" Jul 02 23:06:43.729652 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.729635 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 02 23:06:43.736959 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.736928 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhcv7\" (UniqueName: \"kubernetes.io/projected/b775c85d-e8f4-4c22-ad54-03c160c8e749-kube-api-access-hhcv7\") pod \"cert-manager-operator-controller-manager-55c444bf86-wkfzr\" (UID: \"b775c85d-e8f4-4c22-ad54-03c160c8e749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr" Jul 02 23:06:43.737092 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.736973 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b775c85d-e8f4-4c22-ad54-03c160c8e749-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wkfzr\" (UID: \"b775c85d-e8f4-4c22-ad54-03c160c8e749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr" Jul 02 23:06:43.738316 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.738295 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr"] Jul 02 23:06:43.838219 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.838185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhcv7\" (UniqueName: \"kubernetes.io/projected/b775c85d-e8f4-4c22-ad54-03c160c8e749-kube-api-access-hhcv7\") pod \"cert-manager-operator-controller-manager-55c444bf86-wkfzr\" (UID: \"b775c85d-e8f4-4c22-ad54-03c160c8e749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr" Jul 02 23:06:43.838219 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.838218 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b775c85d-e8f4-4c22-ad54-03c160c8e749-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wkfzr\" (UID: \"b775c85d-e8f4-4c22-ad54-03c160c8e749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr" Jul 02 23:06:43.838621 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.838594 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b775c85d-e8f4-4c22-ad54-03c160c8e749-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wkfzr\" (UID: \"b775c85d-e8f4-4c22-ad54-03c160c8e749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr" Jul 02 23:06:43.853823 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:43.853798 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhcv7\" (UniqueName: \"kubernetes.io/projected/b775c85d-e8f4-4c22-ad54-03c160c8e749-kube-api-access-hhcv7\") pod \"cert-manager-operator-controller-manager-55c444bf86-wkfzr\" (UID: \"b775c85d-e8f4-4c22-ad54-03c160c8e749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr" Jul 02 23:06:44.034809 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:44.034727 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr" Jul 02 23:06:44.172142 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:44.172110 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr"] Jul 02 23:06:44.175671 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:06:44.175644 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb775c85d_e8f4_4c22_ad54_03c160c8e749.slice/crio-fc94d6ba4e665c8ee3ebaec1a88fdb7bd20790517f10ce8697edc8483c33d3c7 WatchSource:0}: Error finding container fc94d6ba4e665c8ee3ebaec1a88fdb7bd20790517f10ce8697edc8483c33d3c7: Status 404 returned error can't find the container with id fc94d6ba4e665c8ee3ebaec1a88fdb7bd20790517f10ce8697edc8483c33d3c7 Jul 02 23:06:44.179131 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:44.179115 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:06:44.793214 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:44.793176 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr" event={"ID":"b775c85d-e8f4-4c22-ad54-03c160c8e749","Type":"ContainerStarted","Data":"fc94d6ba4e665c8ee3ebaec1a88fdb7bd20790517f10ce8697edc8483c33d3c7"} Jul 02 23:06:45.407543 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:45.407511 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:45.464344 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:45.464310 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:46.803676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:46.803585 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr" event={"ID":"b775c85d-e8f4-4c22-ad54-03c160c8e749","Type":"ContainerStarted","Data":"2438b6517c0685bef4f2e22b0acb4ca714ba26219c13d4ec0adf36528f109424"} Jul 02 23:06:46.825649 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:46.825598 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wkfzr" podStartSLOduration=1.474244769 podStartE2EDuration="3.825580729s" totalImagesPullingTime="2.35133596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:06:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:06:46.823374821 +0000 UTC m=+313.896627186" watchObservedRunningTime="2026-07-02 23:06:46.825580729 +0000 UTC m=+313.898833090" Jul 02 23:06:47.174111 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:47.174076 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:47.317848 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:47.317816 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:48.230934 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.230897 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7"] Jul 02 23:06:48.234628 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.234604 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:48.237465 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.237443 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tdjmp\"" Jul 02 23:06:48.237561 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.237527 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 23:06:48.237652 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.237534 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 23:06:48.245335 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.245311 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7"] Jul 02 23:06:48.270930 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.270895 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:48.271104 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.270961 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:48.271104 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.270995 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtpgh\" (UniqueName: \"kubernetes.io/projected/d9914e7f-c6b6-44f7-8e55-74360945b5fa-kube-api-access-jtpgh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:48.372331 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.372287 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:48.372544 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.372357 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:48.372544 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.372400 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtpgh\" (UniqueName: \"kubernetes.io/projected/d9914e7f-c6b6-44f7-8e55-74360945b5fa-kube-api-access-jtpgh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:48.372805 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.372781 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:48.372865 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.372804 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:48.382358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.382332 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtpgh\" (UniqueName: \"kubernetes.io/projected/d9914e7f-c6b6-44f7-8e55-74360945b5fa-kube-api-access-jtpgh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:48.544678 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.544548 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:48.674075 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.674037 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7"] Jul 02 23:06:48.676128 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:06:48.676096 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9914e7f_c6b6_44f7_8e55_74360945b5fa.slice/crio-54cc4e1436a8b517a450685c54bd9bbb0f41b58ae56f3f73972a8df3d74a68ba WatchSource:0}: Error finding container 54cc4e1436a8b517a450685c54bd9bbb0f41b58ae56f3f73972a8df3d74a68ba: Status 404 returned error can't find the container with id 54cc4e1436a8b517a450685c54bd9bbb0f41b58ae56f3f73972a8df3d74a68ba Jul 02 23:06:48.813438 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.813353 2568 generic.go:356] "Generic (PLEG): container finished" podID="d9914e7f-c6b6-44f7-8e55-74360945b5fa" containerID="55b6d43139cdc9fd930ecf73e9894b8a1680a2a4b915dc76932d454bf9c9534d" exitCode=0 Jul 02 23:06:48.813621 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.813430 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" event={"ID":"d9914e7f-c6b6-44f7-8e55-74360945b5fa","Type":"ContainerDied","Data":"55b6d43139cdc9fd930ecf73e9894b8a1680a2a4b915dc76932d454bf9c9534d"} Jul 02 23:06:48.813621 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:48.813480 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" event={"ID":"d9914e7f-c6b6-44f7-8e55-74360945b5fa","Type":"ContainerStarted","Data":"54cc4e1436a8b517a450685c54bd9bbb0f41b58ae56f3f73972a8df3d74a68ba"} Jul 02 23:06:49.491772 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:49.491742 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:49.700426 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:49.700349 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:49.932925 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:49.932885 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-659c9"] Jul 02 23:06:49.937045 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:49.937005 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-659c9" Jul 02 23:06:49.939944 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:49.939917 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 23:06:49.940073 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:49.939921 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 23:06:49.940801 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:49.940752 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xqm4f\"" Jul 02 23:06:49.946391 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:49.946343 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-659c9"] Jul 02 23:06:49.987244 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:49.987098 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/42a08662-f0ef-4496-9b59-15137f754232-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-659c9\" (UID: \"42a08662-f0ef-4496-9b59-15137f754232\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-659c9" Jul 02 23:06:49.987394 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:49.987249 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrdfb\" (UniqueName: \"kubernetes.io/projected/42a08662-f0ef-4496-9b59-15137f754232-kube-api-access-vrdfb\") pod \"cert-manager-cainjector-559cdd8c87-659c9\" (UID: \"42a08662-f0ef-4496-9b59-15137f754232\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-659c9" Jul 02 23:06:50.088757 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:50.088704 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrdfb\" (UniqueName: \"kubernetes.io/projected/42a08662-f0ef-4496-9b59-15137f754232-kube-api-access-vrdfb\") pod \"cert-manager-cainjector-559cdd8c87-659c9\" (UID: \"42a08662-f0ef-4496-9b59-15137f754232\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-659c9" Jul 02 23:06:50.088942 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:50.088785 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/42a08662-f0ef-4496-9b59-15137f754232-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-659c9\" (UID: \"42a08662-f0ef-4496-9b59-15137f754232\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-659c9" Jul 02 23:06:50.098611 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:50.098584 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/42a08662-f0ef-4496-9b59-15137f754232-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-659c9\" (UID: \"42a08662-f0ef-4496-9b59-15137f754232\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-659c9" Jul 02 23:06:50.098867 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:50.098843 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrdfb\" (UniqueName: \"kubernetes.io/projected/42a08662-f0ef-4496-9b59-15137f754232-kube-api-access-vrdfb\") pod \"cert-manager-cainjector-559cdd8c87-659c9\" (UID: \"42a08662-f0ef-4496-9b59-15137f754232\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-659c9" Jul 02 23:06:50.249374 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:50.249290 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-659c9" Jul 02 23:06:50.415598 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:50.415543 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-659c9"] Jul 02 23:06:50.417984 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:06:50.417954 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42a08662_f0ef_4496_9b59_15137f754232.slice/crio-98d7875fe72331cfece64cd97d12c22a7ea9348c2a1c73dcdaa3d263627d2eab WatchSource:0}: Error finding container 98d7875fe72331cfece64cd97d12c22a7ea9348c2a1c73dcdaa3d263627d2eab: Status 404 returned error can't find the container with id 98d7875fe72331cfece64cd97d12c22a7ea9348c2a1c73dcdaa3d263627d2eab Jul 02 23:06:50.824639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:50.824601 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-659c9" event={"ID":"42a08662-f0ef-4496-9b59-15137f754232","Type":"ContainerStarted","Data":"98d7875fe72331cfece64cd97d12c22a7ea9348c2a1c73dcdaa3d263627d2eab"} Jul 02 23:06:51.830399 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:51.830357 2568 generic.go:356] "Generic (PLEG): container finished" podID="d9914e7f-c6b6-44f7-8e55-74360945b5fa" containerID="b0a57124623ad442442e6c7080d92cb7f5f6769de8b721a98adeb377806ca522" exitCode=0 Jul 02 23:06:51.830790 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:51.830443 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" event={"ID":"d9914e7f-c6b6-44f7-8e55-74360945b5fa","Type":"ContainerDied","Data":"b0a57124623ad442442e6c7080d92cb7f5f6769de8b721a98adeb377806ca522"} Jul 02 23:06:52.408597 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:52.408542 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:52.474287 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:52.474249 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:52.838041 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:52.837769 2568 generic.go:356] "Generic (PLEG): container finished" podID="d9914e7f-c6b6-44f7-8e55-74360945b5fa" containerID="625cc7fa2dc8c4d5d77f98aa7ecd195705cdd99553b10dbeddf32cc5190dc842" exitCode=0 Jul 02 23:06:52.838041 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:52.837875 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" event={"ID":"d9914e7f-c6b6-44f7-8e55-74360945b5fa","Type":"ContainerDied","Data":"625cc7fa2dc8c4d5d77f98aa7ecd195705cdd99553b10dbeddf32cc5190dc842"} Jul 02 23:06:53.975520 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:53.975495 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:54.022195 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.022171 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtpgh\" (UniqueName: \"kubernetes.io/projected/d9914e7f-c6b6-44f7-8e55-74360945b5fa-kube-api-access-jtpgh\") pod \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " Jul 02 23:06:54.022312 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.022204 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-util\") pod \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " Jul 02 23:06:54.022312 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.022248 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-bundle\") pod \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\" (UID: \"d9914e7f-c6b6-44f7-8e55-74360945b5fa\") " Jul 02 23:06:54.022634 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.022608 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-bundle" (OuterVolumeSpecName: "bundle") pod "d9914e7f-c6b6-44f7-8e55-74360945b5fa" (UID: "d9914e7f-c6b6-44f7-8e55-74360945b5fa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:06:54.024228 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.024204 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9914e7f-c6b6-44f7-8e55-74360945b5fa-kube-api-access-jtpgh" (OuterVolumeSpecName: "kube-api-access-jtpgh") pod "d9914e7f-c6b6-44f7-8e55-74360945b5fa" (UID: "d9914e7f-c6b6-44f7-8e55-74360945b5fa"). InnerVolumeSpecName "kube-api-access-jtpgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:06:54.027210 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.027191 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-util" (OuterVolumeSpecName: "util") pod "d9914e7f-c6b6-44f7-8e55-74360945b5fa" (UID: "d9914e7f-c6b6-44f7-8e55-74360945b5fa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:06:54.122943 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.122905 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtpgh\" (UniqueName: \"kubernetes.io/projected/d9914e7f-c6b6-44f7-8e55-74360945b5fa-kube-api-access-jtpgh\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:06:54.122943 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.122941 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-util\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:06:54.123140 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.122983 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9914e7f-c6b6-44f7-8e55-74360945b5fa-bundle\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:06:54.849358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.849327 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" event={"ID":"d9914e7f-c6b6-44f7-8e55-74360945b5fa","Type":"ContainerDied","Data":"54cc4e1436a8b517a450685c54bd9bbb0f41b58ae56f3f73972a8df3d74a68ba"} Jul 02 23:06:54.849358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.849357 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f99vf7" Jul 02 23:06:54.849539 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:54.849371 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54cc4e1436a8b517a450685c54bd9bbb0f41b58ae56f3f73972a8df3d74a68ba" Jul 02 23:06:55.467990 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.467964 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zgdfj"] Jul 02 23:06:55.468409 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.468260 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9914e7f-c6b6-44f7-8e55-74360945b5fa" containerName="extract" Jul 02 23:06:55.468409 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.468277 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9914e7f-c6b6-44f7-8e55-74360945b5fa" containerName="extract" Jul 02 23:06:55.468409 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.468294 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9914e7f-c6b6-44f7-8e55-74360945b5fa" containerName="pull" Jul 02 23:06:55.468409 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.468302 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9914e7f-c6b6-44f7-8e55-74360945b5fa" containerName="pull" Jul 02 23:06:55.468409 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.468324 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9914e7f-c6b6-44f7-8e55-74360945b5fa" containerName="util" Jul 02 23:06:55.468409 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.468330 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9914e7f-c6b6-44f7-8e55-74360945b5fa" containerName="util" Jul 02 23:06:55.468409 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.468401 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9914e7f-c6b6-44f7-8e55-74360945b5fa" containerName="extract" Jul 02 23:06:55.472286 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.472269 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" Jul 02 23:06:55.474850 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.474831 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-b57x8\"" Jul 02 23:06:55.479300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.479275 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zgdfj"] Jul 02 23:06:55.535227 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.535197 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8625f5c9-a690-4085-9f20-b3c484d73801-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zgdfj\" (UID: \"8625f5c9-a690-4085-9f20-b3c484d73801\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" Jul 02 23:06:55.535369 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.535236 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz2dr\" (UniqueName: \"kubernetes.io/projected/8625f5c9-a690-4085-9f20-b3c484d73801-kube-api-access-sz2dr\") pod \"cert-manager-webhook-6486ffd997-zgdfj\" (UID: \"8625f5c9-a690-4085-9f20-b3c484d73801\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" Jul 02 23:06:55.635883 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.635849 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8625f5c9-a690-4085-9f20-b3c484d73801-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zgdfj\" (UID: \"8625f5c9-a690-4085-9f20-b3c484d73801\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" Jul 02 23:06:55.635883 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.635886 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz2dr\" (UniqueName: \"kubernetes.io/projected/8625f5c9-a690-4085-9f20-b3c484d73801-kube-api-access-sz2dr\") pod \"cert-manager-webhook-6486ffd997-zgdfj\" (UID: \"8625f5c9-a690-4085-9f20-b3c484d73801\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" Jul 02 23:06:55.644309 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.644279 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8625f5c9-a690-4085-9f20-b3c484d73801-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zgdfj\" (UID: \"8625f5c9-a690-4085-9f20-b3c484d73801\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" Jul 02 23:06:55.644494 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.644476 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz2dr\" (UniqueName: \"kubernetes.io/projected/8625f5c9-a690-4085-9f20-b3c484d73801-kube-api-access-sz2dr\") pod \"cert-manager-webhook-6486ffd997-zgdfj\" (UID: \"8625f5c9-a690-4085-9f20-b3c484d73801\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" Jul 02 23:06:55.782885 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.782793 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" Jul 02 23:06:55.925417 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:55.925394 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zgdfj"] Jul 02 23:06:55.927366 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:06:55.927340 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8625f5c9_a690_4085_9f20_b3c484d73801.slice/crio-ab9974ecef37b07f5df7318878a12155d0fd46c5bb311017e7376464732b165f WatchSource:0}: Error finding container ab9974ecef37b07f5df7318878a12155d0fd46c5bb311017e7376464732b165f: Status 404 returned error can't find the container with id ab9974ecef37b07f5df7318878a12155d0fd46c5bb311017e7376464732b165f Jul 02 23:06:56.700844 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:56.700816 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:56.866498 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:56.866461 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" event={"ID":"8625f5c9-a690-4085-9f20-b3c484d73801","Type":"ContainerStarted","Data":"ab9974ecef37b07f5df7318878a12155d0fd46c5bb311017e7376464732b165f"} Jul 02 23:06:58.671212 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:58.671176 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:58.720855 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:58.720827 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:58.816546 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:58.816517 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:58.877185 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:58.877157 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-659c9" event={"ID":"42a08662-f0ef-4496-9b59-15137f754232","Type":"ContainerStarted","Data":"268740c7dbdcae69c499228d368b4a258f29cc33d018ec6e151c0e1dff9773af"} Jul 02 23:06:58.878740 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:58.878711 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" event={"ID":"8625f5c9-a690-4085-9f20-b3c484d73801","Type":"ContainerStarted","Data":"edcdbedabb2c93975ab66458a285fec7185a2c3e3b95c0051b59000f18ac218c"} Jul 02 23:06:58.878857 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:58.878769 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" Jul 02 23:06:58.892077 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:58.892036 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-659c9" podStartSLOduration=2.43350065 podStartE2EDuration="9.892023572s" totalImagesPullingTime="7.458522922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:06:58.891172041 +0000 UTC m=+325.964424404" watchObservedRunningTime="2026-07-02 23:06:58.892023572 +0000 UTC m=+325.965275936" Jul 02 23:06:58.895645 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:58.895626 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:06:58.908797 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:06:58.908759 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" podStartSLOduration=1.953431143 podStartE2EDuration="3.90874783s" totalImagesPullingTime="1.955316687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:06:58.907348791 +0000 UTC m=+325.980601158" watchObservedRunningTime="2026-07-02 23:06:58.90874783 +0000 UTC m=+325.982000191" Jul 02 23:07:04.884645 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:04.884617 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zgdfj" Jul 02 23:07:05.682579 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:05.682536 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:05.781209 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:05.781185 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:07.778331 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:07.778299 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:08.275610 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.275576 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q"] Jul 02 23:07:08.323819 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.323796 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q"] Jul 02 23:07:08.323946 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.323887 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:08.326732 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.326711 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 23:07:08.326858 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.326772 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 23:07:08.327680 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.327662 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tdjmp\"" Jul 02 23:07:08.435730 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.435704 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:08.435834 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.435736 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:08.435834 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.435817 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdt8b\" (UniqueName: \"kubernetes.io/projected/ecd619f6-8a4b-446f-84de-935460f16e1f-kube-api-access-mdt8b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:08.536098 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.536044 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdt8b\" (UniqueName: \"kubernetes.io/projected/ecd619f6-8a4b-446f-84de-935460f16e1f-kube-api-access-mdt8b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:08.536098 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.536077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:08.536098 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.536098 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:08.536435 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.536421 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:08.536476 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.536453 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:08.543872 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.543851 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdt8b\" (UniqueName: \"kubernetes.io/projected/ecd619f6-8a4b-446f-84de-935460f16e1f-kube-api-access-mdt8b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:08.632554 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.632533 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:08.772912 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.772891 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q"] Jul 02 23:07:08.774787 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:07:08.774757 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecd619f6_8a4b_446f_84de_935460f16e1f.slice/crio-5f80bded8faa639d01ca8ab5c7c7f03867a9348c6c495dabc7ffbc76801c285a WatchSource:0}: Error finding container 5f80bded8faa639d01ca8ab5c7c7f03867a9348c6c495dabc7ffbc76801c285a: Status 404 returned error can't find the container with id 5f80bded8faa639d01ca8ab5c7c7f03867a9348c6c495dabc7ffbc76801c285a Jul 02 23:07:08.921316 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.921286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" event={"ID":"ecd619f6-8a4b-446f-84de-935460f16e1f","Type":"ContainerStarted","Data":"35e0ab243de6f6cf8548b68b76f45cfa0ee1a1ecbbff20d1042931166e5c32f5"} Jul 02 23:07:08.921751 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:08.921341 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" event={"ID":"ecd619f6-8a4b-446f-84de-935460f16e1f","Type":"ContainerStarted","Data":"5f80bded8faa639d01ca8ab5c7c7f03867a9348c6c495dabc7ffbc76801c285a"} Jul 02 23:07:09.926936 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:09.926910 2568 generic.go:356] "Generic (PLEG): container finished" podID="ecd619f6-8a4b-446f-84de-935460f16e1f" containerID="35e0ab243de6f6cf8548b68b76f45cfa0ee1a1ecbbff20d1042931166e5c32f5" exitCode=0 Jul 02 23:07:09.927279 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:09.926985 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" event={"ID":"ecd619f6-8a4b-446f-84de-935460f16e1f","Type":"ContainerDied","Data":"35e0ab243de6f6cf8548b68b76f45cfa0ee1a1ecbbff20d1042931166e5c32f5"} Jul 02 23:07:10.932931 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:10.932903 2568 generic.go:356] "Generic (PLEG): container finished" podID="ecd619f6-8a4b-446f-84de-935460f16e1f" containerID="436f143e02de80bb76e788dd7c3c89bc34a086929b0075f802d522dcae23f2a4" exitCode=0 Jul 02 23:07:10.933299 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:10.932989 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" event={"ID":"ecd619f6-8a4b-446f-84de-935460f16e1f","Type":"ContainerDied","Data":"436f143e02de80bb76e788dd7c3c89bc34a086929b0075f802d522dcae23f2a4"} Jul 02 23:07:11.938343 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:11.938312 2568 generic.go:356] "Generic (PLEG): container finished" podID="ecd619f6-8a4b-446f-84de-935460f16e1f" containerID="f19e5f3e96b5d66a6218fab50f65dce966ae3cffcb7ef40c6170e77a78763fe0" exitCode=0 Jul 02 23:07:11.938759 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:11.938385 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" event={"ID":"ecd619f6-8a4b-446f-84de-935460f16e1f","Type":"ContainerDied","Data":"f19e5f3e96b5d66a6218fab50f65dce966ae3cffcb7ef40c6170e77a78763fe0"} Jul 02 23:07:13.059384 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.059361 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:13.168093 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.168074 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdt8b\" (UniqueName: \"kubernetes.io/projected/ecd619f6-8a4b-446f-84de-935460f16e1f-kube-api-access-mdt8b\") pod \"ecd619f6-8a4b-446f-84de-935460f16e1f\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " Jul 02 23:07:13.168230 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.168109 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-bundle\") pod \"ecd619f6-8a4b-446f-84de-935460f16e1f\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " Jul 02 23:07:13.168230 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.168151 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-util\") pod \"ecd619f6-8a4b-446f-84de-935460f16e1f\" (UID: \"ecd619f6-8a4b-446f-84de-935460f16e1f\") " Jul 02 23:07:13.168843 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.168815 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-bundle" (OuterVolumeSpecName: "bundle") pod "ecd619f6-8a4b-446f-84de-935460f16e1f" (UID: "ecd619f6-8a4b-446f-84de-935460f16e1f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:07:13.170062 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.170027 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecd619f6-8a4b-446f-84de-935460f16e1f-kube-api-access-mdt8b" (OuterVolumeSpecName: "kube-api-access-mdt8b") pod "ecd619f6-8a4b-446f-84de-935460f16e1f" (UID: "ecd619f6-8a4b-446f-84de-935460f16e1f"). InnerVolumeSpecName "kube-api-access-mdt8b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:07:13.173420 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.173399 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-util" (OuterVolumeSpecName: "util") pod "ecd619f6-8a4b-446f-84de-935460f16e1f" (UID: "ecd619f6-8a4b-446f-84de-935460f16e1f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:07:13.269206 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.269155 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdt8b\" (UniqueName: \"kubernetes.io/projected/ecd619f6-8a4b-446f-84de-935460f16e1f-kube-api-access-mdt8b\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:07:13.269206 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.269177 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-bundle\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:07:13.269206 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.269186 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecd619f6-8a4b-446f-84de-935460f16e1f-util\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:07:13.947669 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.947643 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" event={"ID":"ecd619f6-8a4b-446f-84de-935460f16e1f","Type":"ContainerDied","Data":"5f80bded8faa639d01ca8ab5c7c7f03867a9348c6c495dabc7ffbc76801c285a"} Jul 02 23:07:13.947788 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.947688 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f80bded8faa639d01ca8ab5c7c7f03867a9348c6c495dabc7ffbc76801c285a" Jul 02 23:07:13.947788 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:13.947652 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrr2q" Jul 02 23:07:19.621436 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:19.621367 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:19.722348 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:19.722314 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:20.255952 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.255923 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:20.518965 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.518890 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:20.624097 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.624071 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:20.657708 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.657685 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq"] Jul 02 23:07:20.657968 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.657957 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecd619f6-8a4b-446f-84de-935460f16e1f" containerName="extract" Jul 02 23:07:20.658013 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.657970 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecd619f6-8a4b-446f-84de-935460f16e1f" containerName="extract" Jul 02 23:07:20.658013 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.657979 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecd619f6-8a4b-446f-84de-935460f16e1f" containerName="pull" Jul 02 23:07:20.658013 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.657986 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecd619f6-8a4b-446f-84de-935460f16e1f" containerName="pull" Jul 02 23:07:20.658013 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.658005 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecd619f6-8a4b-446f-84de-935460f16e1f" containerName="util" Jul 02 23:07:20.658013 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.658012 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecd619f6-8a4b-446f-84de-935460f16e1f" containerName="util" Jul 02 23:07:20.658155 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.658067 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecd619f6-8a4b-446f-84de-935460f16e1f" containerName="extract" Jul 02 23:07:20.663489 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.663474 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.667909 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.667891 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 23:07:20.668246 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.668230 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 23:07:20.668893 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.668872 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tznn6\"" Jul 02 23:07:20.668996 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.668893 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 23:07:20.672282 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.672264 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 23:07:20.683450 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.683426 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq"] Jul 02 23:07:20.727732 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.727698 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3e845ee6-2bd5-4ef2-bbc0-445166acc977-apiservice-cert\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.727844 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.727739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlqj5\" (UniqueName: \"kubernetes.io/projected/3e845ee6-2bd5-4ef2-bbc0-445166acc977-kube-api-access-qlqj5\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.727844 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.727816 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e845ee6-2bd5-4ef2-bbc0-445166acc977-tmp\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.727941 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.727847 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e845ee6-2bd5-4ef2-bbc0-445166acc977-webhook-cert\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.727941 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.727874 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3e845ee6-2bd5-4ef2-bbc0-445166acc977-manifests\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.780371 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.780315 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:20.828114 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.828093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3e845ee6-2bd5-4ef2-bbc0-445166acc977-apiservice-cert\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.828197 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.828119 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlqj5\" (UniqueName: \"kubernetes.io/projected/3e845ee6-2bd5-4ef2-bbc0-445166acc977-kube-api-access-qlqj5\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.828272 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.828251 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e845ee6-2bd5-4ef2-bbc0-445166acc977-tmp\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.828317 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.828294 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e845ee6-2bd5-4ef2-bbc0-445166acc977-webhook-cert\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.828358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.828333 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3e845ee6-2bd5-4ef2-bbc0-445166acc977-manifests\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.828689 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.828661 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3e845ee6-2bd5-4ef2-bbc0-445166acc977-manifests\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.830467 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.830444 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e845ee6-2bd5-4ef2-bbc0-445166acc977-tmp\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.830602 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.830582 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e845ee6-2bd5-4ef2-bbc0-445166acc977-webhook-cert\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.830725 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.830709 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3e845ee6-2bd5-4ef2-bbc0-445166acc977-apiservice-cert\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.835611 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.835593 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlqj5\" (UniqueName: \"kubernetes.io/projected/3e845ee6-2bd5-4ef2-bbc0-445166acc977-kube-api-access-qlqj5\") pod \"opendatahub-operator-controller-manager-676b9dcdc-q5zgq\" (UID: \"3e845ee6-2bd5-4ef2-bbc0-445166acc977\") " pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.880412 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.880388 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:20.973291 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.973271 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:20.982221 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:20.982198 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:21.081748 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:21.081706 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:21.134007 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:21.133984 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq"] Jul 02 23:07:21.136783 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:07:21.136760 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e845ee6_2bd5_4ef2_bbc0_445166acc977.slice/crio-a66749bb4469b2cd039559858186fdb29eb097c8c8a93ee516968a48474c2a28 WatchSource:0}: Error finding container a66749bb4469b2cd039559858186fdb29eb097c8c8a93ee516968a48474c2a28: Status 404 returned error can't find the container with id a66749bb4469b2cd039559858186fdb29eb097c8c8a93ee516968a48474c2a28 Jul 02 23:07:21.182762 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:21.182740 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:21.984267 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:21.984196 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" event={"ID":"3e845ee6-2bd5-4ef2-bbc0-445166acc977","Type":"ContainerStarted","Data":"a66749bb4469b2cd039559858186fdb29eb097c8c8a93ee516968a48474c2a28"} Jul 02 23:07:24.115899 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:24.115867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:24.219389 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:24.219364 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:24.999012 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:24.998976 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" event={"ID":"3e845ee6-2bd5-4ef2-bbc0-445166acc977","Type":"ContainerStarted","Data":"bb84f5ae48bb0fab0ff3600cba81d8304c718e9805b8c8837f2371186bd85d41"} Jul 02 23:07:26.004643 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:26.004610 2568 generic.go:356] "Generic (PLEG): container finished" podID="3e845ee6-2bd5-4ef2-bbc0-445166acc977" containerID="bb84f5ae48bb0fab0ff3600cba81d8304c718e9805b8c8837f2371186bd85d41" exitCode=0 Jul 02 23:07:26.005034 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:26.004699 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" event={"ID":"3e845ee6-2bd5-4ef2-bbc0-445166acc977","Type":"ContainerDied","Data":"bb84f5ae48bb0fab0ff3600cba81d8304c718e9805b8c8837f2371186bd85d41"} Jul 02 23:07:27.014406 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:27.014355 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" event={"ID":"3e845ee6-2bd5-4ef2-bbc0-445166acc977","Type":"ContainerStarted","Data":"4354486d95938a5112efa7c572a5778f197e3e30c0e8115f7d79b3a36cf3e35c"} Jul 02 23:07:27.014860 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:27.014453 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:27.038646 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:27.038601 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" podStartSLOduration=2.7152056890000003 podStartE2EDuration="7.038586238s" totalImagesPullingTime="3.251908056s" totalInitContainerRuntime="1.071472493s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:07:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:27.03645088 +0000 UTC m=+354.109703264" watchObservedRunningTime="2026-07-02 23:07:27.038586238 +0000 UTC m=+354.111838593" Jul 02 23:07:38.019438 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:38.019411 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-676b9dcdc-q5zgq" Jul 02 23:07:39.734412 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:39.734382 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:39.842068 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:39.842039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:40.498101 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.498069 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:40.679803 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.679777 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5"] Jul 02 23:07:40.688300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.688279 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" Jul 02 23:07:40.692038 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.692014 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 23:07:40.692139 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.692091 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-t8vmt\"" Jul 02 23:07:40.693263 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.693242 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 23:07:40.693358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.693275 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 23:07:40.693358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.693345 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 23:07:40.709021 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.708999 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5"] Jul 02 23:07:40.728349 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.728317 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:40.769727 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.769677 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4xvtf"] Jul 02 23:07:40.772865 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.772849 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:07:40.774457 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.774435 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9a1431f-25ad-48b5-b658-86f757f2f5a1-tmp\") pod \"kube-auth-proxy-99946bbb8-tg2t5\" (UID: \"d9a1431f-25ad-48b5-b658-86f757f2f5a1\") " pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" Jul 02 23:07:40.774546 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.774508 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9a1431f-25ad-48b5-b658-86f757f2f5a1-tls-certs\") pod \"kube-auth-proxy-99946bbb8-tg2t5\" (UID: \"d9a1431f-25ad-48b5-b658-86f757f2f5a1\") " pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" Jul 02 23:07:40.774546 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.774538 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9fjq\" (UniqueName: \"kubernetes.io/projected/d9a1431f-25ad-48b5-b658-86f757f2f5a1-kube-api-access-j9fjq\") pod \"kube-auth-proxy-99946bbb8-tg2t5\" (UID: \"d9a1431f-25ad-48b5-b658-86f757f2f5a1\") " pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" Jul 02 23:07:40.775836 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.775818 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 02 23:07:40.775978 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.775935 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-rllsb\"" Jul 02 23:07:40.786001 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.785980 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4xvtf"] Jul 02 23:07:40.815611 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.815588 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx"] Jul 02 23:07:40.818696 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.818681 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:40.821600 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.821560 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 02 23:07:40.821708 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.821628 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 02 23:07:40.821888 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.821870 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:07:40.821966 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.821917 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kx4jp\"" Jul 02 23:07:40.822026 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.822002 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 23:07:40.822294 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.822279 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 02 23:07:40.831303 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.831282 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx"] Jul 02 23:07:40.862234 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.862214 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:40.874846 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.874822 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9fjq\" (UniqueName: \"kubernetes.io/projected/d9a1431f-25ad-48b5-b658-86f757f2f5a1-kube-api-access-j9fjq\") pod \"kube-auth-proxy-99946bbb8-tg2t5\" (UID: \"d9a1431f-25ad-48b5-b658-86f757f2f5a1\") " pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" Jul 02 23:07:40.874937 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.874862 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9a1431f-25ad-48b5-b658-86f757f2f5a1-tmp\") pod \"kube-auth-proxy-99946bbb8-tg2t5\" (UID: \"d9a1431f-25ad-48b5-b658-86f757f2f5a1\") " pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" Jul 02 23:07:40.874937 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.874897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkxvb\" (UniqueName: \"kubernetes.io/projected/68e8c8db-a45b-47f0-8918-1ece0183f62e-kube-api-access-hkxvb\") pod \"odh-model-controller-7998c67dc4-4xvtf\" (UID: \"68e8c8db-a45b-47f0-8918-1ece0183f62e\") " pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:07:40.875017 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.874963 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8c8db-a45b-47f0-8918-1ece0183f62e-cert\") pod \"odh-model-controller-7998c67dc4-4xvtf\" (UID: \"68e8c8db-a45b-47f0-8918-1ece0183f62e\") " pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:07:40.875017 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.875002 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9a1431f-25ad-48b5-b658-86f757f2f5a1-tls-certs\") pod \"kube-auth-proxy-99946bbb8-tg2t5\" (UID: \"d9a1431f-25ad-48b5-b658-86f757f2f5a1\") " pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" Jul 02 23:07:40.876900 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.876881 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9a1431f-25ad-48b5-b658-86f757f2f5a1-tmp\") pod \"kube-auth-proxy-99946bbb8-tg2t5\" (UID: \"d9a1431f-25ad-48b5-b658-86f757f2f5a1\") " pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" Jul 02 23:07:40.877648 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.877632 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9a1431f-25ad-48b5-b658-86f757f2f5a1-tls-certs\") pod \"kube-auth-proxy-99946bbb8-tg2t5\" (UID: \"d9a1431f-25ad-48b5-b658-86f757f2f5a1\") " pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" Jul 02 23:07:40.883375 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.883354 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9fjq\" (UniqueName: \"kubernetes.io/projected/d9a1431f-25ad-48b5-b658-86f757f2f5a1-kube-api-access-j9fjq\") pod \"kube-auth-proxy-99946bbb8-tg2t5\" (UID: \"d9a1431f-25ad-48b5-b658-86f757f2f5a1\") " pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" Jul 02 23:07:40.957212 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.957180 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:40.976016 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.975995 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8c8db-a45b-47f0-8918-1ece0183f62e-cert\") pod \"odh-model-controller-7998c67dc4-4xvtf\" (UID: \"68e8c8db-a45b-47f0-8918-1ece0183f62e\") " pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:07:40.976112 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.976030 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlhvr\" (UniqueName: \"kubernetes.io/projected/efe92fb8-f483-4e1c-b64d-1711720f5bc2-kube-api-access-qlhvr\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:40.976173 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.976110 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efe92fb8-f483-4e1c-b64d-1711720f5bc2-cert\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:40.976173 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:07:40.976115 2568 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 02 23:07:40.976173 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.976137 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/efe92fb8-f483-4e1c-b64d-1711720f5bc2-metrics-cert\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:40.976173 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.976164 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/efe92fb8-f483-4e1c-b64d-1711720f5bc2-manager-config\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:40.976342 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:07:40.976174 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8c8db-a45b-47f0-8918-1ece0183f62e-cert podName:68e8c8db-a45b-47f0-8918-1ece0183f62e nodeName:}" failed. No retries permitted until 2026-07-02 23:07:41.476160434 +0000 UTC m=+368.549412792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68e8c8db-a45b-47f0-8918-1ece0183f62e-cert") pod "odh-model-controller-7998c67dc4-4xvtf" (UID: "68e8c8db-a45b-47f0-8918-1ece0183f62e") : secret "odh-model-controller-webhook-cert" not found Jul 02 23:07:40.976342 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.976212 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkxvb\" (UniqueName: \"kubernetes.io/projected/68e8c8db-a45b-47f0-8918-1ece0183f62e-kube-api-access-hkxvb\") pod \"odh-model-controller-7998c67dc4-4xvtf\" (UID: \"68e8c8db-a45b-47f0-8918-1ece0183f62e\") " pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:07:40.985051 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.985018 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkxvb\" (UniqueName: \"kubernetes.io/projected/68e8c8db-a45b-47f0-8918-1ece0183f62e-kube-api-access-hkxvb\") pod \"odh-model-controller-7998c67dc4-4xvtf\" (UID: \"68e8c8db-a45b-47f0-8918-1ece0183f62e\") " pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:07:40.997824 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:40.997808 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" Jul 02 23:07:41.058162 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.058125 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:41.077307 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.077276 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efe92fb8-f483-4e1c-b64d-1711720f5bc2-cert\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:41.077440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.077320 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/efe92fb8-f483-4e1c-b64d-1711720f5bc2-metrics-cert\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:41.077440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.077350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/efe92fb8-f483-4e1c-b64d-1711720f5bc2-manager-config\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:41.077440 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.077432 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlhvr\" (UniqueName: \"kubernetes.io/projected/efe92fb8-f483-4e1c-b64d-1711720f5bc2-kube-api-access-qlhvr\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:41.078522 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.078474 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/efe92fb8-f483-4e1c-b64d-1711720f5bc2-manager-config\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:41.080820 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.080784 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/efe92fb8-f483-4e1c-b64d-1711720f5bc2-metrics-cert\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:41.080896 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.080875 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efe92fb8-f483-4e1c-b64d-1711720f5bc2-cert\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:41.085211 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.085190 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlhvr\" (UniqueName: \"kubernetes.io/projected/efe92fb8-f483-4e1c-b64d-1711720f5bc2-kube-api-access-qlhvr\") pod \"lws-controller-manager-7f69f5c6b6-j42gx\" (UID: \"efe92fb8-f483-4e1c-b64d-1711720f5bc2\") " pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:41.125788 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.125754 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5"] Jul 02 23:07:41.126848 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.126830 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:41.127835 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:07:41.127811 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a1431f_25ad_48b5_b658_86f757f2f5a1.slice/crio-5bf63b5a316f69a042bc196f40838617bf476f704363f50492e69870a0c0eff4 WatchSource:0}: Error finding container 5bf63b5a316f69a042bc196f40838617bf476f704363f50492e69870a0c0eff4: Status 404 returned error can't find the container with id 5bf63b5a316f69a042bc196f40838617bf476f704363f50492e69870a0c0eff4 Jul 02 23:07:41.259042 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.259019 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx"] Jul 02 23:07:41.261228 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:07:41.261196 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe92fb8_f483_4e1c_b64d_1711720f5bc2.slice/crio-ca965a50cb8c26404c2e857a9f98a930f687ba34c90736b5ca4136675f4a9837 WatchSource:0}: Error finding container ca965a50cb8c26404c2e857a9f98a930f687ba34c90736b5ca4136675f4a9837: Status 404 returned error can't find the container with id ca965a50cb8c26404c2e857a9f98a930f687ba34c90736b5ca4136675f4a9837 Jul 02 23:07:41.446974 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.446945 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:41.482463 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.482435 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8c8db-a45b-47f0-8918-1ece0183f62e-cert\") pod \"odh-model-controller-7998c67dc4-4xvtf\" (UID: \"68e8c8db-a45b-47f0-8918-1ece0183f62e\") " pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:07:41.482606 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:07:41.482543 2568 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 02 23:07:41.482654 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:07:41.482607 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8c8db-a45b-47f0-8918-1ece0183f62e-cert podName:68e8c8db-a45b-47f0-8918-1ece0183f62e nodeName:}" failed. No retries permitted until 2026-07-02 23:07:42.482594403 +0000 UTC m=+369.555846743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68e8c8db-a45b-47f0-8918-1ece0183f62e-cert") pod "odh-model-controller-7998c67dc4-4xvtf" (UID: "68e8c8db-a45b-47f0-8918-1ece0183f62e") : secret "odh-model-controller-webhook-cert" not found Jul 02 23:07:41.602074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.602049 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:41.915969 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.915939 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:41.983753 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:41.983715 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:42.078952 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:42.078912 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:42.089323 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:42.089255 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" event={"ID":"efe92fb8-f483-4e1c-b64d-1711720f5bc2","Type":"ContainerStarted","Data":"ca965a50cb8c26404c2e857a9f98a930f687ba34c90736b5ca4136675f4a9837"} Jul 02 23:07:42.091999 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:42.091972 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" event={"ID":"d9a1431f-25ad-48b5-b658-86f757f2f5a1","Type":"ContainerStarted","Data":"5bf63b5a316f69a042bc196f40838617bf476f704363f50492e69870a0c0eff4"} Jul 02 23:07:42.492615 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:42.492576 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8c8db-a45b-47f0-8918-1ece0183f62e-cert\") pod \"odh-model-controller-7998c67dc4-4xvtf\" (UID: \"68e8c8db-a45b-47f0-8918-1ece0183f62e\") " pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:07:42.495476 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:42.495444 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68e8c8db-a45b-47f0-8918-1ece0183f62e-cert\") pod \"odh-model-controller-7998c67dc4-4xvtf\" (UID: \"68e8c8db-a45b-47f0-8918-1ece0183f62e\") " pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:07:42.589367 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:42.589328 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:07:43.000236 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:43.000203 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4xvtf"] Jul 02 23:07:43.002656 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:07:43.002616 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e8c8db_a45b_47f0_8918_1ece0183f62e.slice/crio-1951d5f204caa93f6b38ac909203e8863b4dd7c6c4b1291c7df6f6d3154987ef WatchSource:0}: Error finding container 1951d5f204caa93f6b38ac909203e8863b4dd7c6c4b1291c7df6f6d3154987ef: Status 404 returned error can't find the container with id 1951d5f204caa93f6b38ac909203e8863b4dd7c6c4b1291c7df6f6d3154987ef Jul 02 23:07:43.097513 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:43.097479 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" event={"ID":"68e8c8db-a45b-47f0-8918-1ece0183f62e","Type":"ContainerStarted","Data":"1951d5f204caa93f6b38ac909203e8863b4dd7c6c4b1291c7df6f6d3154987ef"} Jul 02 23:07:43.098820 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:43.098793 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" event={"ID":"efe92fb8-f483-4e1c-b64d-1711720f5bc2","Type":"ContainerStarted","Data":"3e9cc626ea3fd61fbe8ec3f6497f9dd29b4da859f93793092c0d73410c15caee"} Jul 02 23:07:43.098931 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:43.098919 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:43.117504 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:43.117465 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" podStartSLOduration=1.4544287439999999 podStartE2EDuration="3.117449672s" totalImagesPullingTime="1.663020928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:43.116164338 +0000 UTC m=+370.189416700" watchObservedRunningTime="2026-07-02 23:07:43.117449672 +0000 UTC m=+370.190702035" Jul 02 23:07:43.120138 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:43.120116 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:44.107266 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:44.107231 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" event={"ID":"d9a1431f-25ad-48b5-b658-86f757f2f5a1","Type":"ContainerStarted","Data":"c23a55798a30e0fd9a8d63fec5fc926927b4e8d6a235b55c456b5ab1934b72a0"} Jul 02 23:07:44.124384 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:44.124334 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-99946bbb8-tg2t5" podStartSLOduration=1.3617349810000001 podStartE2EDuration="4.124316171s" totalImagesPullingTime="2.76258119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:44.123197525 +0000 UTC m=+371.196449902" watchObservedRunningTime="2026-07-02 23:07:44.124316171 +0000 UTC m=+371.197568533" Jul 02 23:07:46.524458 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:46.524379 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:46.624706 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:46.624671 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:47.125522 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:47.125479 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" event={"ID":"68e8c8db-a45b-47f0-8918-1ece0183f62e","Type":"ContainerStarted","Data":"6767f6d3d41e3d075ff5f3851d6cfbc527cc9d0529c7e0f8265286cc6a185b48"} Jul 02 23:07:47.125703 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:47.125679 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:07:47.268413 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:47.268380 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:47.453248 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:47.453179 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:47.621964 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:47.621926 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:51.023443 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:51.023401 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:51.150485 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:51.150445 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:51.244951 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:51.244920 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:51.422028 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:51.422002 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:52.307829 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:52.307799 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:52.419894 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:52.419860 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:52.528780 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:52.528741 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:07:54.109771 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:54.109744 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7f69f5c6b6-j42gx" Jul 02 23:07:54.130785 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:54.130737 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" podStartSLOduration=10.906387379 podStartE2EDuration="14.130723324s" totalImagesPullingTime="3.224335945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:07:47.147223829 +0000 UTC m=+374.220476191" watchObservedRunningTime="2026-07-02 23:07:54.130723324 +0000 UTC m=+381.203975685" Jul 02 23:07:58.132676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:07:58.132650 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-4xvtf" Jul 02 23:08:08.850087 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:08.850052 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:08:08.903449 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:08.903417 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:08:37.726512 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:37.726479 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:08:37.790022 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:37.789990 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:08:41.031133 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:41.031097 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:08:41.096974 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:41.096942 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:08:41.195270 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:41.195244 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:08:56.483673 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.483598 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp"] Jul 02 23:08:56.487140 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.487115 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:08:56.489543 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.489524 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 23:08:56.489706 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.489687 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 23:08:56.490637 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.490621 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zlgg2\"" Jul 02 23:08:56.494731 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.494711 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp"] Jul 02 23:08:56.656109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.656077 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8fmh\" (UniqueName: \"kubernetes.io/projected/7653d4d8-35e7-4339-b475-109fe9b64f2b-kube-api-access-j8fmh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:08:56.656233 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.656133 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:08:56.656233 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.656159 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:08:56.757081 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.757011 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8fmh\" (UniqueName: \"kubernetes.io/projected/7653d4d8-35e7-4339-b475-109fe9b64f2b-kube-api-access-j8fmh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:08:56.757081 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.757075 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:08:56.757198 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.757099 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:08:56.757407 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.757392 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:08:56.757483 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.757464 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:08:56.765012 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.764984 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8fmh\" (UniqueName: \"kubernetes.io/projected/7653d4d8-35e7-4339-b475-109fe9b64f2b-kube-api-access-j8fmh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:08:56.796783 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.796761 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:08:56.921547 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:56.921519 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp"] Jul 02 23:08:56.924854 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:08:56.924830 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7653d4d8_35e7_4339_b475_109fe9b64f2b.slice/crio-97941d8bb6e9d73eec9c52b857fe2a066cbbc0dd48822c4817f40c507bbe373a WatchSource:0}: Error finding container 97941d8bb6e9d73eec9c52b857fe2a066cbbc0dd48822c4817f40c507bbe373a: Status 404 returned error can't find the container with id 97941d8bb6e9d73eec9c52b857fe2a066cbbc0dd48822c4817f40c507bbe373a Jul 02 23:08:57.081396 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.081368 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5"] Jul 02 23:08:57.084870 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.084850 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:08:57.093032 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.093001 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5"] Jul 02 23:08:57.262604 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.262582 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpdtk\" (UniqueName: \"kubernetes.io/projected/6aed00e5-2462-4f86-adb2-4acde41936a5-kube-api-access-tpdtk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:08:57.262722 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.262646 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:08:57.262722 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.262684 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:08:57.363421 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.363400 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:08:57.363531 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.363425 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:08:57.363531 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.363466 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpdtk\" (UniqueName: \"kubernetes.io/projected/6aed00e5-2462-4f86-adb2-4acde41936a5-kube-api-access-tpdtk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:08:57.363873 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.363852 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:08:57.363941 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.363872 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:08:57.371985 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.371961 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpdtk\" (UniqueName: \"kubernetes.io/projected/6aed00e5-2462-4f86-adb2-4acde41936a5-kube-api-access-tpdtk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:08:57.393937 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.393909 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:08:57.453768 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.453734 2568 generic.go:356] "Generic (PLEG): container finished" podID="7653d4d8-35e7-4339-b475-109fe9b64f2b" containerID="2d4e48fc0b6afe437ee055f30a6026e59a22d865fb00364d220e2b8cc90abd54" exitCode=0 Jul 02 23:08:57.453873 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.453797 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" event={"ID":"7653d4d8-35e7-4339-b475-109fe9b64f2b","Type":"ContainerDied","Data":"2d4e48fc0b6afe437ee055f30a6026e59a22d865fb00364d220e2b8cc90abd54"} Jul 02 23:08:57.453873 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.453849 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" event={"ID":"7653d4d8-35e7-4339-b475-109fe9b64f2b","Type":"ContainerStarted","Data":"97941d8bb6e9d73eec9c52b857fe2a066cbbc0dd48822c4817f40c507bbe373a"} Jul 02 23:08:57.520232 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.520208 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5"] Jul 02 23:08:57.521887 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:08:57.521860 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aed00e5_2462_4f86_adb2_4acde41936a5.slice/crio-fd460b201df72adcca0a2165d6035f65ccbe5c698a02750266df2b2562d8bfbc WatchSource:0}: Error finding container fd460b201df72adcca0a2165d6035f65ccbe5c698a02750266df2b2562d8bfbc: Status 404 returned error can't find the container with id fd460b201df72adcca0a2165d6035f65ccbe5c698a02750266df2b2562d8bfbc Jul 02 23:08:57.681642 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.681612 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw"] Jul 02 23:08:57.685000 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.684984 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:08:57.692813 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.692792 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw"] Jul 02 23:08:57.767461 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.767438 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjsb6\" (UniqueName: \"kubernetes.io/projected/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-kube-api-access-jjsb6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:08:57.767609 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.767470 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:08:57.767609 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.767551 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:08:57.868669 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.868621 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjsb6\" (UniqueName: \"kubernetes.io/projected/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-kube-api-access-jjsb6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:08:57.868669 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.868653 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:08:57.868804 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.868695 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:08:57.868960 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.868943 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:08:57.869002 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.868984 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:08:57.876959 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.876934 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjsb6\" (UniqueName: \"kubernetes.io/projected/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-kube-api-access-jjsb6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:08:57.993924 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:57.993905 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:08:58.090950 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.090851 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz"] Jul 02 23:08:58.094553 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.094533 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:08:58.105709 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.105683 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz"] Jul 02 23:08:58.122249 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.122229 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw"] Jul 02 23:08:58.142817 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:08:58.142787 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a19ad93_b86a_47e7_8e1d_c8172c6dd9a2.slice/crio-6b53ca482558a4e3ae76d572fa21edbd489260c5714fa49e09ad5d55d5bd1d71 WatchSource:0}: Error finding container 6b53ca482558a4e3ae76d572fa21edbd489260c5714fa49e09ad5d55d5bd1d71: Status 404 returned error can't find the container with id 6b53ca482558a4e3ae76d572fa21edbd489260c5714fa49e09ad5d55d5bd1d71 Jul 02 23:08:58.273184 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.273147 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:08:58.273273 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.273214 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:08:58.273351 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.273328 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffkhk\" (UniqueName: \"kubernetes.io/projected/73ef1068-e9f1-47cf-8449-ceac273f427e-kube-api-access-ffkhk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:08:58.374349 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.374292 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffkhk\" (UniqueName: \"kubernetes.io/projected/73ef1068-e9f1-47cf-8449-ceac273f427e-kube-api-access-ffkhk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:08:58.374349 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.374338 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:08:58.374502 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.374363 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:08:58.374664 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.374646 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:08:58.374727 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.374695 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:08:58.382705 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.382689 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffkhk\" (UniqueName: \"kubernetes.io/projected/73ef1068-e9f1-47cf-8449-ceac273f427e-kube-api-access-ffkhk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:08:58.407659 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.407642 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:08:58.469903 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.469851 2568 generic.go:356] "Generic (PLEG): container finished" podID="7653d4d8-35e7-4339-b475-109fe9b64f2b" containerID="2fd64d7c59ae4cf6de76317dd8fd0b643629d50725b7f897bf720fc6a4f3679c" exitCode=0 Jul 02 23:08:58.470070 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.469919 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" event={"ID":"7653d4d8-35e7-4339-b475-109fe9b64f2b","Type":"ContainerDied","Data":"2fd64d7c59ae4cf6de76317dd8fd0b643629d50725b7f897bf720fc6a4f3679c"} Jul 02 23:08:58.472049 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.472020 2568 generic.go:356] "Generic (PLEG): container finished" podID="0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" containerID="541b8958184c43622f16458818784e221d7cc6cc52290cd48468ca851cbcc3d5" exitCode=0 Jul 02 23:08:58.472164 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.472102 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" event={"ID":"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2","Type":"ContainerDied","Data":"541b8958184c43622f16458818784e221d7cc6cc52290cd48468ca851cbcc3d5"} Jul 02 23:08:58.472164 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.472150 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" event={"ID":"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2","Type":"ContainerStarted","Data":"6b53ca482558a4e3ae76d572fa21edbd489260c5714fa49e09ad5d55d5bd1d71"} Jul 02 23:08:58.474047 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.474022 2568 generic.go:356] "Generic (PLEG): container finished" podID="6aed00e5-2462-4f86-adb2-4acde41936a5" containerID="33d2fbd05287453b9f0db15a3b9ff426dd528d2403fa057d64dd595bdfcd7ef5" exitCode=0 Jul 02 23:08:58.474152 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.474090 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" event={"ID":"6aed00e5-2462-4f86-adb2-4acde41936a5","Type":"ContainerDied","Data":"33d2fbd05287453b9f0db15a3b9ff426dd528d2403fa057d64dd595bdfcd7ef5"} Jul 02 23:08:58.474152 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.474147 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" event={"ID":"6aed00e5-2462-4f86-adb2-4acde41936a5","Type":"ContainerStarted","Data":"fd460b201df72adcca0a2165d6035f65ccbe5c698a02750266df2b2562d8bfbc"} Jul 02 23:08:58.548903 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:58.548871 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz"] Jul 02 23:08:58.550717 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:08:58.550693 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ef1068_e9f1_47cf_8449_ceac273f427e.slice/crio-a635ac22352cd9470b1f793610faeb59b214ac0014ed3ac15b88dc3361ef52df WatchSource:0}: Error finding container a635ac22352cd9470b1f793610faeb59b214ac0014ed3ac15b88dc3361ef52df: Status 404 returned error can't find the container with id a635ac22352cd9470b1f793610faeb59b214ac0014ed3ac15b88dc3361ef52df Jul 02 23:08:59.480418 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:59.480351 2568 generic.go:356] "Generic (PLEG): container finished" podID="73ef1068-e9f1-47cf-8449-ceac273f427e" containerID="3bfa5b786550a8d4e6e48039a290af1e53af403fbcabf82d3f9f7696a92f6a9a" exitCode=0 Jul 02 23:08:59.480615 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:59.480439 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" event={"ID":"73ef1068-e9f1-47cf-8449-ceac273f427e","Type":"ContainerDied","Data":"3bfa5b786550a8d4e6e48039a290af1e53af403fbcabf82d3f9f7696a92f6a9a"} Jul 02 23:08:59.480615 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:59.480491 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" event={"ID":"73ef1068-e9f1-47cf-8449-ceac273f427e","Type":"ContainerStarted","Data":"a635ac22352cd9470b1f793610faeb59b214ac0014ed3ac15b88dc3361ef52df"} Jul 02 23:08:59.482163 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:59.482141 2568 generic.go:356] "Generic (PLEG): container finished" podID="0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" containerID="265a9ead3e0a1df66e617c69a2209678ce208d543170bc19eaef9ff671f3d9a2" exitCode=0 Jul 02 23:08:59.482255 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:59.482231 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" event={"ID":"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2","Type":"ContainerDied","Data":"265a9ead3e0a1df66e617c69a2209678ce208d543170bc19eaef9ff671f3d9a2"} Jul 02 23:08:59.484365 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:59.484348 2568 generic.go:356] "Generic (PLEG): container finished" podID="7653d4d8-35e7-4339-b475-109fe9b64f2b" containerID="e4799293eb39945abd8e4945bd60a88fa5edef3e6a6f8e959e056d324d1415ed" exitCode=0 Jul 02 23:08:59.484426 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:08:59.484409 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" event={"ID":"7653d4d8-35e7-4339-b475-109fe9b64f2b","Type":"ContainerDied","Data":"e4799293eb39945abd8e4945bd60a88fa5edef3e6a6f8e959e056d324d1415ed"} Jul 02 23:09:00.491609 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.491542 2568 generic.go:356] "Generic (PLEG): container finished" podID="73ef1068-e9f1-47cf-8449-ceac273f427e" containerID="40a42d2cd70ceec163caf982d46efe0a177b84b00b674d5b1227a853d7e55a3e" exitCode=0 Jul 02 23:09:00.491959 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.491628 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" event={"ID":"73ef1068-e9f1-47cf-8449-ceac273f427e","Type":"ContainerDied","Data":"40a42d2cd70ceec163caf982d46efe0a177b84b00b674d5b1227a853d7e55a3e"} Jul 02 23:09:00.493869 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.493848 2568 generic.go:356] "Generic (PLEG): container finished" podID="0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" containerID="7674b137bd6e5a9b9bb482e2ea9b83622f489ed555981b172c6bc8f2290c0e24" exitCode=0 Jul 02 23:09:00.493958 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.493926 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" event={"ID":"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2","Type":"ContainerDied","Data":"7674b137bd6e5a9b9bb482e2ea9b83622f489ed555981b172c6bc8f2290c0e24"} Jul 02 23:09:00.495680 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.495660 2568 generic.go:356] "Generic (PLEG): container finished" podID="6aed00e5-2462-4f86-adb2-4acde41936a5" containerID="0ff5444b64dfaed6ea851294b87216d2e21541aba3cdef0ad5b3eb076c4059d5" exitCode=0 Jul 02 23:09:00.495771 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.495691 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" event={"ID":"6aed00e5-2462-4f86-adb2-4acde41936a5","Type":"ContainerDied","Data":"0ff5444b64dfaed6ea851294b87216d2e21541aba3cdef0ad5b3eb076c4059d5"} Jul 02 23:09:00.623434 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.623406 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:09:00.792794 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.792767 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-bundle\") pod \"7653d4d8-35e7-4339-b475-109fe9b64f2b\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " Jul 02 23:09:00.792880 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.792851 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-util\") pod \"7653d4d8-35e7-4339-b475-109fe9b64f2b\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " Jul 02 23:09:00.792924 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.792881 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8fmh\" (UniqueName: \"kubernetes.io/projected/7653d4d8-35e7-4339-b475-109fe9b64f2b-kube-api-access-j8fmh\") pod \"7653d4d8-35e7-4339-b475-109fe9b64f2b\" (UID: \"7653d4d8-35e7-4339-b475-109fe9b64f2b\") " Jul 02 23:09:00.793249 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.793222 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-bundle" (OuterVolumeSpecName: "bundle") pod "7653d4d8-35e7-4339-b475-109fe9b64f2b" (UID: "7653d4d8-35e7-4339-b475-109fe9b64f2b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:09:00.794917 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.794897 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7653d4d8-35e7-4339-b475-109fe9b64f2b-kube-api-access-j8fmh" (OuterVolumeSpecName: "kube-api-access-j8fmh") pod "7653d4d8-35e7-4339-b475-109fe9b64f2b" (UID: "7653d4d8-35e7-4339-b475-109fe9b64f2b"). InnerVolumeSpecName "kube-api-access-j8fmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:09:00.798541 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.798520 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-util" (OuterVolumeSpecName: "util") pod "7653d4d8-35e7-4339-b475-109fe9b64f2b" (UID: "7653d4d8-35e7-4339-b475-109fe9b64f2b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:09:00.894091 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.894067 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-util\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:00.894091 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.894089 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8fmh\" (UniqueName: \"kubernetes.io/projected/7653d4d8-35e7-4339-b475-109fe9b64f2b-kube-api-access-j8fmh\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:00.894206 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:00.894103 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7653d4d8-35e7-4339-b475-109fe9b64f2b-bundle\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:01.501297 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.501265 2568 generic.go:356] "Generic (PLEG): container finished" podID="73ef1068-e9f1-47cf-8449-ceac273f427e" containerID="12359eaed05c15cb653137517fcc9451f26e23939650d7b92da528c6d49fb212" exitCode=0 Jul 02 23:09:01.501688 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.501342 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" event={"ID":"73ef1068-e9f1-47cf-8449-ceac273f427e","Type":"ContainerDied","Data":"12359eaed05c15cb653137517fcc9451f26e23939650d7b92da528c6d49fb212"} Jul 02 23:09:01.503418 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.503398 2568 generic.go:356] "Generic (PLEG): container finished" podID="6aed00e5-2462-4f86-adb2-4acde41936a5" containerID="30a5a744cc878fb69f40a6eb07be627e03c42323aa4b59955a825e4cf0624a7a" exitCode=0 Jul 02 23:09:01.503508 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.503476 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" event={"ID":"6aed00e5-2462-4f86-adb2-4acde41936a5","Type":"ContainerDied","Data":"30a5a744cc878fb69f40a6eb07be627e03c42323aa4b59955a825e4cf0624a7a"} Jul 02 23:09:01.505189 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.505166 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" event={"ID":"7653d4d8-35e7-4339-b475-109fe9b64f2b","Type":"ContainerDied","Data":"97941d8bb6e9d73eec9c52b857fe2a066cbbc0dd48822c4817f40c507bbe373a"} Jul 02 23:09:01.505257 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.505205 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97941d8bb6e9d73eec9c52b857fe2a066cbbc0dd48822c4817f40c507bbe373a" Jul 02 23:09:01.505334 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.505321 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp" Jul 02 23:09:01.636133 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.636111 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:09:01.799493 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.799427 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-bundle\") pod \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " Jul 02 23:09:01.799493 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.799469 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-util\") pod \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " Jul 02 23:09:01.799901 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.799522 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjsb6\" (UniqueName: \"kubernetes.io/projected/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-kube-api-access-jjsb6\") pod \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\" (UID: \"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2\") " Jul 02 23:09:01.800023 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.800000 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-bundle" (OuterVolumeSpecName: "bundle") pod "0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" (UID: "0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:09:01.801591 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.801535 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-kube-api-access-jjsb6" (OuterVolumeSpecName: "kube-api-access-jjsb6") pod "0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" (UID: "0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2"). InnerVolumeSpecName "kube-api-access-jjsb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:09:01.805251 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.805219 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-util" (OuterVolumeSpecName: "util") pod "0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" (UID: "0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:09:01.900174 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.900152 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-bundle\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:01.900174 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.900172 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-util\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:01.900303 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:01.900181 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjsb6\" (UniqueName: \"kubernetes.io/projected/0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2-kube-api-access-jjsb6\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:02.513135 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.513100 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" Jul 02 23:09:02.513506 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.513101 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw" event={"ID":"0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2","Type":"ContainerDied","Data":"6b53ca482558a4e3ae76d572fa21edbd489260c5714fa49e09ad5d55d5bd1d71"} Jul 02 23:09:02.513506 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.513219 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b53ca482558a4e3ae76d572fa21edbd489260c5714fa49e09ad5d55d5bd1d71" Jul 02 23:09:02.665954 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.665935 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:09:02.667863 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.667844 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:09:02.806834 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.806766 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-bundle\") pod \"73ef1068-e9f1-47cf-8449-ceac273f427e\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " Jul 02 23:09:02.806834 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.806809 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-util\") pod \"73ef1068-e9f1-47cf-8449-ceac273f427e\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " Jul 02 23:09:02.807022 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.806864 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-bundle\") pod \"6aed00e5-2462-4f86-adb2-4acde41936a5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " Jul 02 23:09:02.807022 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.806894 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffkhk\" (UniqueName: \"kubernetes.io/projected/73ef1068-e9f1-47cf-8449-ceac273f427e-kube-api-access-ffkhk\") pod \"73ef1068-e9f1-47cf-8449-ceac273f427e\" (UID: \"73ef1068-e9f1-47cf-8449-ceac273f427e\") " Jul 02 23:09:02.807022 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.806929 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpdtk\" (UniqueName: \"kubernetes.io/projected/6aed00e5-2462-4f86-adb2-4acde41936a5-kube-api-access-tpdtk\") pod \"6aed00e5-2462-4f86-adb2-4acde41936a5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " Jul 02 23:09:02.807022 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.806983 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-util\") pod \"6aed00e5-2462-4f86-adb2-4acde41936a5\" (UID: \"6aed00e5-2462-4f86-adb2-4acde41936a5\") " Jul 02 23:09:02.807404 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.807377 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-bundle" (OuterVolumeSpecName: "bundle") pod "73ef1068-e9f1-47cf-8449-ceac273f427e" (UID: "73ef1068-e9f1-47cf-8449-ceac273f427e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:09:02.807594 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.807385 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-bundle" (OuterVolumeSpecName: "bundle") pod "6aed00e5-2462-4f86-adb2-4acde41936a5" (UID: "6aed00e5-2462-4f86-adb2-4acde41936a5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:09:02.809080 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.809050 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aed00e5-2462-4f86-adb2-4acde41936a5-kube-api-access-tpdtk" (OuterVolumeSpecName: "kube-api-access-tpdtk") pod "6aed00e5-2462-4f86-adb2-4acde41936a5" (UID: "6aed00e5-2462-4f86-adb2-4acde41936a5"). InnerVolumeSpecName "kube-api-access-tpdtk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:09:02.809176 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.809063 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73ef1068-e9f1-47cf-8449-ceac273f427e-kube-api-access-ffkhk" (OuterVolumeSpecName: "kube-api-access-ffkhk") pod "73ef1068-e9f1-47cf-8449-ceac273f427e" (UID: "73ef1068-e9f1-47cf-8449-ceac273f427e"). InnerVolumeSpecName "kube-api-access-ffkhk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:09:02.815684 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.815659 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-util" (OuterVolumeSpecName: "util") pod "73ef1068-e9f1-47cf-8449-ceac273f427e" (UID: "73ef1068-e9f1-47cf-8449-ceac273f427e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:09:02.815830 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.815809 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-util" (OuterVolumeSpecName: "util") pod "6aed00e5-2462-4f86-adb2-4acde41936a5" (UID: "6aed00e5-2462-4f86-adb2-4acde41936a5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:09:02.907926 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.907904 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-util\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:02.907926 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.907924 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-bundle\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:02.908031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.907932 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73ef1068-e9f1-47cf-8449-ceac273f427e-util\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:02.908031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.907940 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6aed00e5-2462-4f86-adb2-4acde41936a5-bundle\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:02.908031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.907948 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffkhk\" (UniqueName: \"kubernetes.io/projected/73ef1068-e9f1-47cf-8449-ceac273f427e-kube-api-access-ffkhk\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:02.908031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:02.907959 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpdtk\" (UniqueName: \"kubernetes.io/projected/6aed00e5-2462-4f86-adb2-4acde41936a5-kube-api-access-tpdtk\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:09:03.520230 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:03.520209 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" Jul 02 23:09:03.520548 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:03.520240 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5" event={"ID":"6aed00e5-2462-4f86-adb2-4acde41936a5","Type":"ContainerDied","Data":"fd460b201df72adcca0a2165d6035f65ccbe5c698a02750266df2b2562d8bfbc"} Jul 02 23:09:03.520548 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:03.520285 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd460b201df72adcca0a2165d6035f65ccbe5c698a02750266df2b2562d8bfbc" Jul 02 23:09:03.522350 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:03.522327 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" event={"ID":"73ef1068-e9f1-47cf-8449-ceac273f427e","Type":"ContainerDied","Data":"a635ac22352cd9470b1f793610faeb59b214ac0014ed3ac15b88dc3361ef52df"} Jul 02 23:09:03.522434 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:03.522368 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a635ac22352cd9470b1f793610faeb59b214ac0014ed3ac15b88dc3361ef52df" Jul 02 23:09:03.522476 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:03.522434 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz" Jul 02 23:09:10.149411 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:10.149375 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:11.426115 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:11.426079 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:11.527332 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:11.527300 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:13.390065 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:13.390030 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:13.498098 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:13.498071 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:14.345373 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:14.345341 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:14.699993 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:14.699961 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:14.804024 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:14.803998 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:16.737369 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:16.737332 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:16.982519 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:16.982487 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:17.345133 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:17.345100 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:20.746089 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:20.746056 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:21.797687 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:21.797651 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:21.902301 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:21.902265 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:23.790645 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:23.790612 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:25.576421 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:25.576389 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:25.756402 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:25.756372 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:25.883053 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:25.883020 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:25.969393 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:25.969369 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:26.965708 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:26.965676 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:27.063732 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:27.063704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:30.746868 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:30.746838 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:30.849935 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:30.849902 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:33.211964 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:33.211936 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:33.347338 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:33.347312 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:34.940953 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:34.940922 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:35.057481 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:35.057451 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:35.347977 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:35.347905 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:35.446335 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:35.446305 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:39.133691 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:39.133660 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:39.234494 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:39.234465 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:45.684515 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:45.684479 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:45.787645 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:45.787619 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:46.830346 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:46.830298 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:46.898384 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:46.898355 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:48.499954 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:48.499922 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:48.600102 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:48.600073 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:53.291297 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:53.291269 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:53.391630 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:53.391603 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:57.298780 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:57.298750 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:09:57.358880 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:09:57.358857 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:06.898363 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:06.898328 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:06.995152 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:06.995124 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:10.698827 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:10.698794 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:10.798767 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:10.798739 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:15.095494 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:15.095462 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:15.151139 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:15.151108 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:15.494237 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:15.494206 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:15.595026 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:15.594999 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:18.898761 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:18.898726 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:18.960157 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:18.960109 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:24.136401 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:24.136365 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:24.207605 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:24.207548 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:25.270957 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:25.270929 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:39.884727 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:39.884687 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:39.999664 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:39.999632 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:40.068182 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.068150 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:40.297167 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297093 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:10:40.297463 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297437 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73ef1068-e9f1-47cf-8449-ceac273f427e" containerName="util" Jul 02 23:10:40.297463 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297454 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="73ef1068-e9f1-47cf-8449-ceac273f427e" containerName="util" Jul 02 23:10:40.297463 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297466 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aed00e5-2462-4f86-adb2-4acde41936a5" containerName="pull" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297474 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aed00e5-2462-4f86-adb2-4acde41936a5" containerName="pull" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297483 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7653d4d8-35e7-4339-b475-109fe9b64f2b" containerName="pull" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297488 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7653d4d8-35e7-4339-b475-109fe9b64f2b" containerName="pull" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297498 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7653d4d8-35e7-4339-b475-109fe9b64f2b" containerName="extract" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297506 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7653d4d8-35e7-4339-b475-109fe9b64f2b" containerName="extract" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297519 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" containerName="util" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297527 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" containerName="util" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297536 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7653d4d8-35e7-4339-b475-109fe9b64f2b" containerName="util" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297544 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7653d4d8-35e7-4339-b475-109fe9b64f2b" containerName="util" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297557 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73ef1068-e9f1-47cf-8449-ceac273f427e" containerName="pull" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297579 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="73ef1068-e9f1-47cf-8449-ceac273f427e" containerName="pull" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297590 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73ef1068-e9f1-47cf-8449-ceac273f427e" containerName="extract" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297598 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="73ef1068-e9f1-47cf-8449-ceac273f427e" containerName="extract" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297609 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aed00e5-2462-4f86-adb2-4acde41936a5" containerName="extract" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297618 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aed00e5-2462-4f86-adb2-4acde41936a5" containerName="extract" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297636 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" containerName="extract" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297645 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" containerName="extract" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297653 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" containerName="pull" Jul 02 23:10:40.297661 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297661 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" containerName="pull" Jul 02 23:10:40.298188 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297681 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aed00e5-2462-4f86-adb2-4acde41936a5" containerName="util" Jul 02 23:10:40.298188 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297689 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aed00e5-2462-4f86-adb2-4acde41936a5" containerName="util" Jul 02 23:10:40.298188 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297782 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aed00e5-2462-4f86-adb2-4acde41936a5" containerName="extract" Jul 02 23:10:40.298188 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297791 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7653d4d8-35e7-4339-b475-109fe9b64f2b" containerName="extract" Jul 02 23:10:40.298188 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297801 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2" containerName="extract" Jul 02 23:10:40.298188 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.297807 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="73ef1068-e9f1-47cf-8449-ceac273f427e" containerName="extract" Jul 02 23:10:40.300765 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.300751 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" Jul 02 23:10:40.304311 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.304284 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zlgg2\"" Jul 02 23:10:40.304701 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.304300 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 23:10:40.305355 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.305333 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 23:10:40.305465 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.305441 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 23:10:40.307921 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.307899 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:10:40.341963 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.341930 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:10:40.394359 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.394332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxhnl\" (UniqueName: \"kubernetes.io/projected/f7024c9e-cd75-47fc-8b56-526b422b7194-kube-api-access-mxhnl\") pod \"limitador-limitador-54d6d988d5-b64kb\" (UID: \"f7024c9e-cd75-47fc-8b56-526b422b7194\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" Jul 02 23:10:40.394497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.394401 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f7024c9e-cd75-47fc-8b56-526b422b7194-config-file\") pod \"limitador-limitador-54d6d988d5-b64kb\" (UID: \"f7024c9e-cd75-47fc-8b56-526b422b7194\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" Jul 02 23:10:40.495380 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.495354 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxhnl\" (UniqueName: \"kubernetes.io/projected/f7024c9e-cd75-47fc-8b56-526b422b7194-kube-api-access-mxhnl\") pod \"limitador-limitador-54d6d988d5-b64kb\" (UID: \"f7024c9e-cd75-47fc-8b56-526b422b7194\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" Jul 02 23:10:40.495524 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.495418 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f7024c9e-cd75-47fc-8b56-526b422b7194-config-file\") pod \"limitador-limitador-54d6d988d5-b64kb\" (UID: \"f7024c9e-cd75-47fc-8b56-526b422b7194\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" Jul 02 23:10:40.495968 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.495949 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f7024c9e-cd75-47fc-8b56-526b422b7194-config-file\") pod \"limitador-limitador-54d6d988d5-b64kb\" (UID: \"f7024c9e-cd75-47fc-8b56-526b422b7194\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" Jul 02 23:10:40.503557 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.503540 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxhnl\" (UniqueName: \"kubernetes.io/projected/f7024c9e-cd75-47fc-8b56-526b422b7194-kube-api-access-mxhnl\") pod \"limitador-limitador-54d6d988d5-b64kb\" (UID: \"f7024c9e-cd75-47fc-8b56-526b422b7194\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" Jul 02 23:10:40.612327 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.612286 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" Jul 02 23:10:40.721979 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.721945 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-bbcc2"] Jul 02 23:10:40.725460 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.725440 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bbcc2" Jul 02 23:10:40.729100 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.729082 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dckhd\"" Jul 02 23:10:40.744484 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.744464 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-bbcc2"] Jul 02 23:10:40.762981 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.762960 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:10:40.765992 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:10:40.765958 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7024c9e_cd75_47fc_8b56_526b422b7194.slice/crio-36f3f37a96289500b38732a755145096a33236832c51662bb2d54c5e1a59b927 WatchSource:0}: Error finding container 36f3f37a96289500b38732a755145096a33236832c51662bb2d54c5e1a59b927: Status 404 returned error can't find the container with id 36f3f37a96289500b38732a755145096a33236832c51662bb2d54c5e1a59b927 Jul 02 23:10:40.798422 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.798392 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7wpf\" (UniqueName: \"kubernetes.io/projected/7a4eb349-f785-429b-94a1-3af4589e21b0-kube-api-access-x7wpf\") pod \"authorino-58cd955675-bbcc2\" (UID: \"7a4eb349-f785-429b-94a1-3af4589e21b0\") " pod="kuadrant-system/authorino-58cd955675-bbcc2" Jul 02 23:10:40.801787 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.801766 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:40.892160 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.892039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:40.899224 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.899202 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7wpf\" (UniqueName: \"kubernetes.io/projected/7a4eb349-f785-429b-94a1-3af4589e21b0-kube-api-access-x7wpf\") pod \"authorino-58cd955675-bbcc2\" (UID: \"7a4eb349-f785-429b-94a1-3af4589e21b0\") " pod="kuadrant-system/authorino-58cd955675-bbcc2" Jul 02 23:10:40.910845 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:40.910817 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7wpf\" (UniqueName: \"kubernetes.io/projected/7a4eb349-f785-429b-94a1-3af4589e21b0-kube-api-access-x7wpf\") pod \"authorino-58cd955675-bbcc2\" (UID: \"7a4eb349-f785-429b-94a1-3af4589e21b0\") " pod="kuadrant-system/authorino-58cd955675-bbcc2" Jul 02 23:10:41.000248 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:41.000214 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" event={"ID":"f7024c9e-cd75-47fc-8b56-526b422b7194","Type":"ContainerStarted","Data":"36f3f37a96289500b38732a755145096a33236832c51662bb2d54c5e1a59b927"} Jul 02 23:10:41.034041 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:41.034013 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bbcc2" Jul 02 23:10:41.161779 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:41.161744 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-bbcc2"] Jul 02 23:10:41.163198 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:10:41.163169 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a4eb349_f785_429b_94a1_3af4589e21b0.slice/crio-1dab9218dccdb22e8b0e581c1712cc9ee41f8a9498fd1efe2055d552b1e070a0 WatchSource:0}: Error finding container 1dab9218dccdb22e8b0e581c1712cc9ee41f8a9498fd1efe2055d552b1e070a0: Status 404 returned error can't find the container with id 1dab9218dccdb22e8b0e581c1712cc9ee41f8a9498fd1efe2055d552b1e070a0 Jul 02 23:10:42.008984 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:42.008945 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bbcc2" event={"ID":"7a4eb349-f785-429b-94a1-3af4589e21b0","Type":"ContainerStarted","Data":"1dab9218dccdb22e8b0e581c1712cc9ee41f8a9498fd1efe2055d552b1e070a0"} Jul 02 23:10:46.032980 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:46.032936 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bbcc2" event={"ID":"7a4eb349-f785-429b-94a1-3af4589e21b0","Type":"ContainerStarted","Data":"3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12"} Jul 02 23:10:46.034641 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:46.034607 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" event={"ID":"f7024c9e-cd75-47fc-8b56-526b422b7194","Type":"ContainerStarted","Data":"a8a51e200c082d11bb680c041ad13cf0b5ed05bcf23e63235263baffedae43e6"} Jul 02 23:10:46.034765 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:46.034687 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" Jul 02 23:10:46.049378 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:46.049334 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-bbcc2" podStartSLOduration=2.098644776 podStartE2EDuration="6.04931985s" totalImagesPullingTime="3.950675074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:46.047483938 +0000 UTC m=+553.120736302" watchObservedRunningTime="2026-07-02 23:10:46.04931985 +0000 UTC m=+553.122572192" Jul 02 23:10:46.067837 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:46.067784 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" podStartSLOduration=1.7199928249999998 podStartE2EDuration="6.067765081s" totalImagesPullingTime="4.347772256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:46.066681013 +0000 UTC m=+553.139933377" watchObservedRunningTime="2026-07-02 23:10:46.067765081 +0000 UTC m=+553.141017443" Jul 02 23:10:46.076267 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:46.076239 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-bbcc2"] Jul 02 23:10:48.043993 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:48.043955 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-bbcc2" podUID="7a4eb349-f785-429b-94a1-3af4589e21b0" containerName="authorino" containerID="cri-o://3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12" gracePeriod=30 Jul 02 23:10:48.289549 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:48.289525 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bbcc2" Jul 02 23:10:48.465581 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:48.465547 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7wpf\" (UniqueName: \"kubernetes.io/projected/7a4eb349-f785-429b-94a1-3af4589e21b0-kube-api-access-x7wpf\") pod \"7a4eb349-f785-429b-94a1-3af4589e21b0\" (UID: \"7a4eb349-f785-429b-94a1-3af4589e21b0\") " Jul 02 23:10:48.467508 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:48.467488 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a4eb349-f785-429b-94a1-3af4589e21b0-kube-api-access-x7wpf" (OuterVolumeSpecName: "kube-api-access-x7wpf") pod "7a4eb349-f785-429b-94a1-3af4589e21b0" (UID: "7a4eb349-f785-429b-94a1-3af4589e21b0"). InnerVolumeSpecName "kube-api-access-x7wpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:10:48.566389 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:48.566350 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7wpf\" (UniqueName: \"kubernetes.io/projected/7a4eb349-f785-429b-94a1-3af4589e21b0-kube-api-access-x7wpf\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:10:49.055666 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:49.055628 2568 generic.go:356] "Generic (PLEG): container finished" podID="7a4eb349-f785-429b-94a1-3af4589e21b0" containerID="3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12" exitCode=0 Jul 02 23:10:49.056084 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:49.055683 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bbcc2" Jul 02 23:10:49.056084 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:49.055709 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bbcc2" event={"ID":"7a4eb349-f785-429b-94a1-3af4589e21b0","Type":"ContainerDied","Data":"3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12"} Jul 02 23:10:49.056084 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:49.055764 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bbcc2" event={"ID":"7a4eb349-f785-429b-94a1-3af4589e21b0","Type":"ContainerDied","Data":"1dab9218dccdb22e8b0e581c1712cc9ee41f8a9498fd1efe2055d552b1e070a0"} Jul 02 23:10:49.056084 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:49.055786 2568 scope.go:117] "RemoveContainer" containerID="3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12" Jul 02 23:10:49.067837 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:49.067809 2568 scope.go:117] "RemoveContainer" containerID="3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12" Jul 02 23:10:49.068075 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:10:49.068058 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12\": container with ID starting with 3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12 not found: ID does not exist" containerID="3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12" Jul 02 23:10:49.068140 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:49.068081 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12"} err="failed to get container status \"3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12\": rpc error: code = NotFound desc = could not find container \"3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12\": container with ID starting with 3845b810edc41c12ea1e90ea5ebf2ade68794168704f13a899c4c98dcf69cc12 not found: ID does not exist" Jul 02 23:10:49.076976 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:49.076954 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-bbcc2"] Jul 02 23:10:49.081075 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:49.081052 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-bbcc2"] Jul 02 23:10:49.468250 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:49.468216 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a4eb349-f785-429b-94a1-3af4589e21b0" path="/var/lib/kubelet/pods/7a4eb349-f785-429b-94a1-3af4589e21b0/volumes" Jul 02 23:10:54.699435 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:54.699394 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:10:57.040683 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:10:57.040651 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b64kb" Jul 02 23:11:00.021790 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:00.021753 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:00.081043 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:00.081009 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:03.333220 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:03.333192 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:05.407684 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:05.407651 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:05.509544 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:05.509516 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:16.482151 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.482117 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:16.703470 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.703438 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4n9fh"] Jul 02 23:11:16.703894 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.703879 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a4eb349-f785-429b-94a1-3af4589e21b0" containerName="authorino" Jul 02 23:11:16.703894 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.703896 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a4eb349-f785-429b-94a1-3af4589e21b0" containerName="authorino" Jul 02 23:11:16.703987 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.703958 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a4eb349-f785-429b-94a1-3af4589e21b0" containerName="authorino" Jul 02 23:11:16.708898 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.708880 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4n9fh" Jul 02 23:11:16.711379 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.711360 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dckhd\"" Jul 02 23:11:16.714633 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.714611 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4n9fh"] Jul 02 23:11:16.790983 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.790902 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4dh4\" (UniqueName: \"kubernetes.io/projected/deabb0a5-c04c-4b7d-ab00-de429a5835f8-kube-api-access-m4dh4\") pod \"authorino-5d4b67c446-4n9fh\" (UID: \"deabb0a5-c04c-4b7d-ab00-de429a5835f8\") " pod="kuadrant-system/authorino-5d4b67c446-4n9fh" Jul 02 23:11:16.793808 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.793782 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:16.892235 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.892197 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4dh4\" (UniqueName: \"kubernetes.io/projected/deabb0a5-c04c-4b7d-ab00-de429a5835f8-kube-api-access-m4dh4\") pod \"authorino-5d4b67c446-4n9fh\" (UID: \"deabb0a5-c04c-4b7d-ab00-de429a5835f8\") " pod="kuadrant-system/authorino-5d4b67c446-4n9fh" Jul 02 23:11:16.900077 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.900047 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4dh4\" (UniqueName: \"kubernetes.io/projected/deabb0a5-c04c-4b7d-ab00-de429a5835f8-kube-api-access-m4dh4\") pod \"authorino-5d4b67c446-4n9fh\" (UID: \"deabb0a5-c04c-4b7d-ab00-de429a5835f8\") " pod="kuadrant-system/authorino-5d4b67c446-4n9fh" Jul 02 23:11:16.922154 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.922128 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4n9fh"] Jul 02 23:11:16.922397 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:16.922383 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4n9fh" Jul 02 23:11:17.056175 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.056149 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4n9fh"] Jul 02 23:11:17.058059 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:11:17.058031 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddeabb0a5_c04c_4b7d_ab00_de429a5835f8.slice/crio-62e80e5cb705c149332561b86cb23a02c7f968c7aef2ec6c7c6a3e539cf2265b WatchSource:0}: Error finding container 62e80e5cb705c149332561b86cb23a02c7f968c7aef2ec6c7c6a3e539cf2265b: Status 404 returned error can't find the container with id 62e80e5cb705c149332561b86cb23a02c7f968c7aef2ec6c7c6a3e539cf2265b Jul 02 23:11:17.098971 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.098943 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-647445b9b-2xvtn"] Jul 02 23:11:17.106383 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.106362 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-647445b9b-2xvtn" Jul 02 23:11:17.109448 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.109430 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 02 23:11:17.111010 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.110989 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-647445b9b-2xvtn"] Jul 02 23:11:17.194672 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.194631 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-tls-cert\") pod \"authorino-647445b9b-2xvtn\" (UID: \"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed\") " pod="kuadrant-system/authorino-647445b9b-2xvtn" Jul 02 23:11:17.194824 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.194699 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn4mj\" (UniqueName: \"kubernetes.io/projected/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-kube-api-access-xn4mj\") pod \"authorino-647445b9b-2xvtn\" (UID: \"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed\") " pod="kuadrant-system/authorino-647445b9b-2xvtn" Jul 02 23:11:17.196938 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.196912 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4n9fh" event={"ID":"deabb0a5-c04c-4b7d-ab00-de429a5835f8","Type":"ContainerStarted","Data":"62e80e5cb705c149332561b86cb23a02c7f968c7aef2ec6c7c6a3e539cf2265b"} Jul 02 23:11:17.296147 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.296107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-tls-cert\") pod \"authorino-647445b9b-2xvtn\" (UID: \"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed\") " pod="kuadrant-system/authorino-647445b9b-2xvtn" Jul 02 23:11:17.296331 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.296253 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn4mj\" (UniqueName: \"kubernetes.io/projected/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-kube-api-access-xn4mj\") pod \"authorino-647445b9b-2xvtn\" (UID: \"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed\") " pod="kuadrant-system/authorino-647445b9b-2xvtn" Jul 02 23:11:17.298629 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.298594 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-tls-cert\") pod \"authorino-647445b9b-2xvtn\" (UID: \"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed\") " pod="kuadrant-system/authorino-647445b9b-2xvtn" Jul 02 23:11:17.304439 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.304413 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn4mj\" (UniqueName: \"kubernetes.io/projected/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-kube-api-access-xn4mj\") pod \"authorino-647445b9b-2xvtn\" (UID: \"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed\") " pod="kuadrant-system/authorino-647445b9b-2xvtn" Jul 02 23:11:17.416278 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.416247 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-647445b9b-2xvtn" Jul 02 23:11:17.555852 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:17.555826 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-647445b9b-2xvtn"] Jul 02 23:11:17.557307 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:11:17.557277 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fd4cb52_81a4_4e8e_a7e2_4107d482c5ed.slice/crio-e39e02d5f567e4f4a3f38163aa32e1d3008432219e96c3a7d6f838b7e123b24f WatchSource:0}: Error finding container e39e02d5f567e4f4a3f38163aa32e1d3008432219e96c3a7d6f838b7e123b24f: Status 404 returned error can't find the container with id e39e02d5f567e4f4a3f38163aa32e1d3008432219e96c3a7d6f838b7e123b24f Jul 02 23:11:18.203450 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:18.203420 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-647445b9b-2xvtn" event={"ID":"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed","Type":"ContainerStarted","Data":"e39e02d5f567e4f4a3f38163aa32e1d3008432219e96c3a7d6f838b7e123b24f"} Jul 02 23:11:18.204964 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:18.204939 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4n9fh" event={"ID":"deabb0a5-c04c-4b7d-ab00-de429a5835f8","Type":"ContainerStarted","Data":"a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86"} Jul 02 23:11:18.205072 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:18.205034 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-4n9fh" podUID="deabb0a5-c04c-4b7d-ab00-de429a5835f8" containerName="authorino" containerID="cri-o://a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86" gracePeriod=30 Jul 02 23:11:18.220620 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:18.220580 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-4n9fh" podStartSLOduration=1.704318605 podStartE2EDuration="2.220550253s" totalImagesPullingTime="516.231648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:18.218742219 +0000 UTC m=+585.291994582" watchObservedRunningTime="2026-07-02 23:11:18.220550253 +0000 UTC m=+585.293802615" Jul 02 23:11:18.484540 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:18.484513 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4n9fh" Jul 02 23:11:18.608017 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:18.607981 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4dh4\" (UniqueName: \"kubernetes.io/projected/deabb0a5-c04c-4b7d-ab00-de429a5835f8-kube-api-access-m4dh4\") pod \"deabb0a5-c04c-4b7d-ab00-de429a5835f8\" (UID: \"deabb0a5-c04c-4b7d-ab00-de429a5835f8\") " Jul 02 23:11:18.609977 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:18.609954 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deabb0a5-c04c-4b7d-ab00-de429a5835f8-kube-api-access-m4dh4" (OuterVolumeSpecName: "kube-api-access-m4dh4") pod "deabb0a5-c04c-4b7d-ab00-de429a5835f8" (UID: "deabb0a5-c04c-4b7d-ab00-de429a5835f8"). InnerVolumeSpecName "kube-api-access-m4dh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:11:18.709506 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:18.709417 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4dh4\" (UniqueName: \"kubernetes.io/projected/deabb0a5-c04c-4b7d-ab00-de429a5835f8-kube-api-access-m4dh4\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:11:19.211283 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.211245 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-647445b9b-2xvtn" event={"ID":"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed","Type":"ContainerStarted","Data":"f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e"} Jul 02 23:11:19.212418 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.212397 2568 generic.go:356] "Generic (PLEG): container finished" podID="deabb0a5-c04c-4b7d-ab00-de429a5835f8" containerID="a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86" exitCode=0 Jul 02 23:11:19.212524 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.212438 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4n9fh" Jul 02 23:11:19.212524 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.212455 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4n9fh" event={"ID":"deabb0a5-c04c-4b7d-ab00-de429a5835f8","Type":"ContainerDied","Data":"a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86"} Jul 02 23:11:19.212524 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.212481 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4n9fh" event={"ID":"deabb0a5-c04c-4b7d-ab00-de429a5835f8","Type":"ContainerDied","Data":"62e80e5cb705c149332561b86cb23a02c7f968c7aef2ec6c7c6a3e539cf2265b"} Jul 02 23:11:19.212524 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.212497 2568 scope.go:117] "RemoveContainer" containerID="a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86" Jul 02 23:11:19.221037 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.221009 2568 scope.go:117] "RemoveContainer" containerID="a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86" Jul 02 23:11:19.221280 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:11:19.221257 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86\": container with ID starting with a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86 not found: ID does not exist" containerID="a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86" Jul 02 23:11:19.221340 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.221287 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86"} err="failed to get container status \"a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86\": rpc error: code = NotFound desc = could not find container \"a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86\": container with ID starting with a30ebc65eff2396534be616f19091a23b70346fc44d565966681ef128acc2c86 not found: ID does not exist" Jul 02 23:11:19.228123 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.228078 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-647445b9b-2xvtn" podStartSLOduration=1.655024939 podStartE2EDuration="2.228064435s" totalImagesPullingTime="573.039496ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:19.226877424 +0000 UTC m=+586.300129788" watchObservedRunningTime="2026-07-02 23:11:19.228064435 +0000 UTC m=+586.301316797" Jul 02 23:11:19.245847 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.245815 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4n9fh"] Jul 02 23:11:19.251436 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.251404 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4n9fh"] Jul 02 23:11:19.468612 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:19.468517 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="deabb0a5-c04c-4b7d-ab00-de429a5835f8" path="/var/lib/kubelet/pods/deabb0a5-c04c-4b7d-ab00-de429a5835f8/volumes" Jul 02 23:11:23.399672 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:23.399639 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:23.500774 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:23.500743 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:23.604087 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:23.604055 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:23.659553 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:23.659469 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:25.842632 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:25.842596 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:25.936802 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:25.936770 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:33.383727 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:33.383696 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:11:33.384154 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:33.383846 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:11:57.130807 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:57.130730 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:58.218713 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:58.218677 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:11:58.242266 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:11:58.242226 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:12:07.383508 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:07.383478 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:12:07.487815 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:07.487785 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:12:16.310864 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:16.310828 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:12:16.688968 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:16.688934 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:12:23.019768 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:23.019734 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:12:23.695605 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:23.695550 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:12:26.591608 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:26.591553 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:12:26.658226 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:26.658192 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:12:34.983332 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:34.983295 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:12:35.995484 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:35.995441 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:12:41.217228 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:41.217191 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:12:42.692105 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:42.692068 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:12:43.309383 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:43.309351 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:12:48.690813 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:48.690774 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:12:51.597792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.597745 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2"] Jul 02 23:12:51.598279 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.598259 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="deabb0a5-c04c-4b7d-ab00-de429a5835f8" containerName="authorino" Jul 02 23:12:51.598355 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.598283 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="deabb0a5-c04c-4b7d-ab00-de429a5835f8" containerName="authorino" Jul 02 23:12:51.598413 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.598377 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="deabb0a5-c04c-4b7d-ab00-de429a5835f8" containerName="authorino" Jul 02 23:12:51.601477 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.601454 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.604376 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.604348 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 23:12:51.605277 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.605257 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 23:12:51.605362 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.605261 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 02 23:12:51.605362 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.605348 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pgtwg\"" Jul 02 23:12:51.610425 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.610401 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2"] Jul 02 23:12:51.616848 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.616822 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:12:51.692037 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.691998 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.692215 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.692052 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.692215 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.692139 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c0699c-1740-4355-982e-e5fde171b0a4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.692215 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.692169 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z24p8\" (UniqueName: \"kubernetes.io/projected/d9c0699c-1740-4355-982e-e5fde171b0a4-kube-api-access-z24p8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.692215 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.692204 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.692368 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.692234 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.793682 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.793643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.793873 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.793692 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c0699c-1740-4355-982e-e5fde171b0a4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.793873 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.793729 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z24p8\" (UniqueName: \"kubernetes.io/projected/d9c0699c-1740-4355-982e-e5fde171b0a4-kube-api-access-z24p8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.793873 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.793765 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.793873 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.793781 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.793873 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.793843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.794131 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.794060 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.794131 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.794111 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.794203 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.794171 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.796139 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.796117 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c0699c-1740-4355-982e-e5fde171b0a4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.796300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.796284 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c0699c-1740-4355-982e-e5fde171b0a4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.802787 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.802768 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z24p8\" (UniqueName: \"kubernetes.io/projected/d9c0699c-1740-4355-982e-e5fde171b0a4-kube-api-access-z24p8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2\" (UID: \"d9c0699c-1740-4355-982e-e5fde171b0a4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:51.911225 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:51.911188 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:12:52.046933 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:52.046907 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2"] Jul 02 23:12:52.048931 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:12:52.048903 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9c0699c_1740_4355_982e_e5fde171b0a4.slice/crio-8549fbc0dc593a44b4667e0e3ed4538102e6fb98ed70ec9572acf5329e8fe242 WatchSource:0}: Error finding container 8549fbc0dc593a44b4667e0e3ed4538102e6fb98ed70ec9572acf5329e8fe242: Status 404 returned error can't find the container with id 8549fbc0dc593a44b4667e0e3ed4538102e6fb98ed70ec9572acf5329e8fe242 Jul 02 23:12:52.050222 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:52.050202 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:12:52.693934 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:52.693869 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" event={"ID":"d9c0699c-1740-4355-982e-e5fde171b0a4","Type":"ContainerStarted","Data":"8549fbc0dc593a44b4667e0e3ed4538102e6fb98ed70ec9572acf5329e8fe242"} Jul 02 23:12:53.293240 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:53.293200 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:12:53.703112 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:53.703069 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" event={"ID":"d9c0699c-1740-4355-982e-e5fde171b0a4","Type":"ContainerStarted","Data":"4389b6fc120dd136c38ccbb3ac2bfda97ccf3cbdd3df31dc135d7a5f91b2ef7b"} Jul 02 23:12:53.703797 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:53.703772 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:12:53.732141 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:12:53.732078 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" podStartSLOduration=1.599416508 podStartE2EDuration="2.732060002s" totalImagesPullingTime="1.132643494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:53.730103948 +0000 UTC m=+680.803356314" watchObservedRunningTime="2026-07-02 23:12:53.732060002 +0000 UTC m=+680.805312366" Jul 02 23:13:01.911743 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:13:01.911704 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:13:01.912123 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:13:01.911760 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:13:01.924084 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:13:01.924055 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:13:02.759675 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:13:02.759644 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2" Jul 02 23:13:23.870329 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:13:23.870284 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:13:23.925834 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:13:23.925801 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:14:03.835265 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:03.835229 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7444d59b49-q9z55"] Jul 02 23:14:03.838619 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:03.838602 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7444d59b49-q9z55" Jul 02 23:14:03.846484 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:03.846457 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7444d59b49-q9z55"] Jul 02 23:14:03.913712 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:03.913680 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmfx6\" (UniqueName: \"kubernetes.io/projected/dabbd3d9-a17b-4ab8-b536-5f15d4955247-kube-api-access-vmfx6\") pod \"authorino-7444d59b49-q9z55\" (UID: \"dabbd3d9-a17b-4ab8-b536-5f15d4955247\") " pod="kuadrant-system/authorino-7444d59b49-q9z55" Jul 02 23:14:03.913895 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:03.913723 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dabbd3d9-a17b-4ab8-b536-5f15d4955247-tls-cert\") pod \"authorino-7444d59b49-q9z55\" (UID: \"dabbd3d9-a17b-4ab8-b536-5f15d4955247\") " pod="kuadrant-system/authorino-7444d59b49-q9z55" Jul 02 23:14:04.014559 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:04.014527 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmfx6\" (UniqueName: \"kubernetes.io/projected/dabbd3d9-a17b-4ab8-b536-5f15d4955247-kube-api-access-vmfx6\") pod \"authorino-7444d59b49-q9z55\" (UID: \"dabbd3d9-a17b-4ab8-b536-5f15d4955247\") " pod="kuadrant-system/authorino-7444d59b49-q9z55" Jul 02 23:14:04.014700 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:04.014587 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dabbd3d9-a17b-4ab8-b536-5f15d4955247-tls-cert\") pod \"authorino-7444d59b49-q9z55\" (UID: \"dabbd3d9-a17b-4ab8-b536-5f15d4955247\") " pod="kuadrant-system/authorino-7444d59b49-q9z55" Jul 02 23:14:04.017002 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:04.016979 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dabbd3d9-a17b-4ab8-b536-5f15d4955247-tls-cert\") pod \"authorino-7444d59b49-q9z55\" (UID: \"dabbd3d9-a17b-4ab8-b536-5f15d4955247\") " pod="kuadrant-system/authorino-7444d59b49-q9z55" Jul 02 23:14:04.022715 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:04.022690 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmfx6\" (UniqueName: \"kubernetes.io/projected/dabbd3d9-a17b-4ab8-b536-5f15d4955247-kube-api-access-vmfx6\") pod \"authorino-7444d59b49-q9z55\" (UID: \"dabbd3d9-a17b-4ab8-b536-5f15d4955247\") " pod="kuadrant-system/authorino-7444d59b49-q9z55" Jul 02 23:14:04.148352 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:04.148332 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7444d59b49-q9z55" Jul 02 23:14:04.252051 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:04.252019 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:14:04.283093 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:04.283066 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7444d59b49-q9z55"] Jul 02 23:14:04.283923 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:14:04.283883 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddabbd3d9_a17b_4ab8_b536_5f15d4955247.slice/crio-ba9cea737d44d3cab20c985fbe6ec5fb98f02c3a2077b484368e07eca86369f3 WatchSource:0}: Error finding container ba9cea737d44d3cab20c985fbe6ec5fb98f02c3a2077b484368e07eca86369f3: Status 404 returned error can't find the container with id ba9cea737d44d3cab20c985fbe6ec5fb98f02c3a2077b484368e07eca86369f3 Jul 02 23:14:04.352031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:04.352001 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:14:05.067247 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.067207 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7444d59b49-q9z55" event={"ID":"dabbd3d9-a17b-4ab8-b536-5f15d4955247","Type":"ContainerStarted","Data":"9c5eeebe2f39889e2e6876c67f81be4b76689101138aaf69c098d845c64df802"} Jul 02 23:14:05.067643 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.067260 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7444d59b49-q9z55" event={"ID":"dabbd3d9-a17b-4ab8-b536-5f15d4955247","Type":"ContainerStarted","Data":"ba9cea737d44d3cab20c985fbe6ec5fb98f02c3a2077b484368e07eca86369f3"} Jul 02 23:14:05.089755 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.089634 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7444d59b49-q9z55" podStartSLOduration=1.5563850970000002 podStartE2EDuration="2.089613708s" totalImagesPullingTime="533.228611ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:14:05.086489118 +0000 UTC m=+752.159741481" watchObservedRunningTime="2026-07-02 23:14:05.089613708 +0000 UTC m=+752.162866072" Jul 02 23:14:05.119229 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.119193 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-647445b9b-2xvtn"] Jul 02 23:14:05.119422 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.119393 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-647445b9b-2xvtn" podUID="4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed" containerName="authorino" containerID="cri-o://f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e" gracePeriod=30 Jul 02 23:14:05.374348 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.374323 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-647445b9b-2xvtn" Jul 02 23:14:05.427281 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.427250 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn4mj\" (UniqueName: \"kubernetes.io/projected/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-kube-api-access-xn4mj\") pod \"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed\" (UID: \"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed\") " Jul 02 23:14:05.427434 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.427291 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-tls-cert\") pod \"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed\" (UID: \"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed\") " Jul 02 23:14:05.429662 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.429629 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:14:05.429816 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.429707 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-kube-api-access-xn4mj" (OuterVolumeSpecName: "kube-api-access-xn4mj") pod "4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed" (UID: "4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed"). InnerVolumeSpecName "kube-api-access-xn4mj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:14:05.437244 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.437214 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed" (UID: "4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:14:05.535900 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.535844 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:14:05.542166 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.542131 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn4mj\" (UniqueName: \"kubernetes.io/projected/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-kube-api-access-xn4mj\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:14:05.542166 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:05.542167 2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed-tls-cert\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:14:06.074226 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:06.074187 2568 generic.go:356] "Generic (PLEG): container finished" podID="4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed" containerID="f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e" exitCode=0 Jul 02 23:14:06.074749 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:06.074253 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-647445b9b-2xvtn" Jul 02 23:14:06.074749 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:06.074275 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-647445b9b-2xvtn" event={"ID":"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed","Type":"ContainerDied","Data":"f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e"} Jul 02 23:14:06.074749 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:06.074320 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-647445b9b-2xvtn" event={"ID":"4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed","Type":"ContainerDied","Data":"e39e02d5f567e4f4a3f38163aa32e1d3008432219e96c3a7d6f838b7e123b24f"} Jul 02 23:14:06.074749 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:06.074338 2568 scope.go:117] "RemoveContainer" containerID="f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e" Jul 02 23:14:06.084118 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:06.084101 2568 scope.go:117] "RemoveContainer" containerID="f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e" Jul 02 23:14:06.084381 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:14:06.084363 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e\": container with ID starting with f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e not found: ID does not exist" containerID="f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e" Jul 02 23:14:06.084442 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:06.084396 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e"} err="failed to get container status \"f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e\": rpc error: code = NotFound desc = could not find container \"f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e\": container with ID starting with f6d9e4f8c4d7ca64a0139dc01107a26ae7f7e69b7b64f829007b83de9629744e not found: ID does not exist" Jul 02 23:14:06.090690 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:06.090666 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-647445b9b-2xvtn"] Jul 02 23:14:06.094099 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:06.094080 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-647445b9b-2xvtn"] Jul 02 23:14:07.468091 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:07.468058 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed" path="/var/lib/kubelet/pods/4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed/volumes" Jul 02 23:14:24.808861 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:24.808826 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:14:30.385219 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:30.385184 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:14:31.696794 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:31.696760 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:14:42.488198 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:42.488162 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:14:52.594530 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:14:52.594490 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:15:01.499268 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:15:01.499177 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:15:12.696415 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:15:12.696383 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:15:14.873801 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:15:14.873762 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:15:16.474305 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:15:16.474261 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:15:31.316315 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:15:31.316272 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:15:40.400664 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:15:40.400631 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:15:50.701388 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:15:50.701353 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:15:56.601389 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:15:56.601358 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:15:56.849639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:15:56.849601 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:15:56.948451 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:15:56.948400 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:15:57.055375 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:15:57.055350 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:16:02.761045 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:02.761013 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:16:02.822659 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:02.822631 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:16:02.898346 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:02.898298 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:16:02.993430 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:02.993389 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:16:03.110723 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:03.110639 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:16:07.687389 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:07.687352 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:16:08.669908 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:08.669872 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:16:08.738241 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:08.738209 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:16:25.394029 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:25.393998 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:16:33.416648 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:33.416622 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:16:33.418801 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:33.418781 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:16:35.094985 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:35.094953 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:16:44.789913 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:44.789874 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:16:54.696830 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:16:54.696795 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:17:05.496622 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:17:05.496590 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:17:14.797705 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:17:14.797672 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:17:27.195385 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:17:27.195351 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:17:36.305756 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:17:36.305721 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:17:58.799833 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:17:58.799749 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:18:09.286000 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:18:09.285963 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:18:29.198258 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:18:29.198222 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:18:38.799507 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:18:38.799473 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:18:54.490599 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:18:54.490550 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:19:03.094246 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:19:03.094208 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:19:13.190078 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:19:13.190041 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:19:22.287188 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:19:22.287157 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:19:29.903522 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:19:29.903446 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:19:31.294924 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:19:31.294878 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:19:31.407628 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:19:31.407595 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:21:20.893397 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:21:20.893316 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:21:31.888337 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:21:31.888304 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:21:33.455720 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:21:33.455689 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:21:33.458622 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:21:33.458602 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:21:42.994012 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:21:42.993971 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:21:51.399598 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:21:51.398740 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:22:00.996031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:22:00.995998 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:22:08.495599 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:22:08.495548 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:22:18.993269 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:22:18.993222 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:22:28.903465 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:22:28.903382 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:22:38.598354 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:22:38.598313 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:22:43.666981 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:22:43.666940 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:22:49.592797 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:22:49.592761 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:23:05.269304 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:23:05.269267 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:23:24.563907 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:23:24.563867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:23:24.696831 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:23:24.696802 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:23:33.093680 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:23:33.093635 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:23:43.398879 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:23:43.398842 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:23:54.489168 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:23:54.489114 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:24:09.901120 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:24:09.901041 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:24:18.492133 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:24:18.492091 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:24:26.891195 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:24:26.891162 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:24:27.894986 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:24:27.894945 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:24:27.946308 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:24:27.946262 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:24:36.092015 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:24:36.091977 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:24:44.991298 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:24:44.991255 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:24:55.091506 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:24:55.091472 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:25:07.699895 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:25:07.699861 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:25:17.299324 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:25:17.299287 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:25:28.793253 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:25:28.793153 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:25:40.492822 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:25:40.492792 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:25:51.693366 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:25:51.693329 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:25:58.998203 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:25:58.998162 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:26:09.997373 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:09.997329 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:26:21.791466 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:21.791432 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:26:32.498562 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:32.498526 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:26:33.488700 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:33.488672 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:26:33.493675 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:33.493656 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:26:36.700730 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:36.700688 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:26:40.691847 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:40.691811 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:26:45.295111 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:45.295067 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:26:48.990885 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:48.990849 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:26:53.551219 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:53.551181 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:26:57.591410 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:57.591315 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:26:58.638766 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:26:58.638725 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:01.368749 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:01.368710 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:01.462528 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:01.462490 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:01.806911 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:01.806822 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:06.099141 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:06.099102 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:27:07.487814 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:07.487780 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:07.588162 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:07.588124 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:07.695477 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:07.695444 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:07.762326 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:07.762240 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:08.874335 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:08.874301 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:08.965102 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:08.965068 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:09.037485 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:09.037452 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:09.137634 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:09.137536 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:09.431735 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:09.431639 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:09.531606 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:09.531578 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:11.286433 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:11.286405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:13.988171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:13.988132 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:14.159495 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:14.159464 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:14.257503 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:14.257402 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:14.445981 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:14.445950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:17.532035 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:17.532006 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:19.954523 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:19.954485 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:20.017496 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:20.017454 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:20.170358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:20.170321 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:20.240358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:20.240293 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:21.440837 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:21.440805 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:21.513849 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:21.513804 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:21.589665 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:21.589623 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:21.816953 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:21.816860 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:22.005312 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:22.005282 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:22.106121 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:22.106032 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:23.546754 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:23.546716 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:24.098723 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:24.098691 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:27:26.563826 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:26.563790 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:26.881735 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:26.881704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:32.820117 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:32.820027 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:32.920583 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:32.920542 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:33.718881 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:33.718847 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:27:34.538388 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:34.538359 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:38.491626 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:38.491590 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:38.557160 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:38.557088 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:38.691454 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:38.691420 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:38.791030 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:38.790951 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:39.893321 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:39.893286 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:40.015458 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:40.015426 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:40.113148 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:40.113115 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:40.798746 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:40.798709 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:40.874667 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:40.874631 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:41.725207 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:41.725177 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:42.490037 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:42.490002 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:27:44.342157 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:44.342124 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:44.560976 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:44.560941 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:44.664513 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:44.664484 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:44.785886 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:44.785848 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:47.814173 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:47.814144 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:50.490104 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:50.490071 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:27:50.691391 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:50.691357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:50.791729 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:50.791659 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:51.197997 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:51.197967 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:51.357510 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:51.357464 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:52.704601 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:52.704547 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:58.296979 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:58.296949 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:58.399308 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:58.399276 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:58.497364 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:58.497324 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:58.598099 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:58.598024 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:27:58.969822 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:27:58.969790 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:28:10.711098 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:28:10.711058 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:28:10.810876 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:28:10.810845 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:28:24.916261 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:28:24.916211 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:28:30.485779 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:28:30.485695 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:28:30.593977 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:28:30.593941 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:28:30.686959 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:28:30.686924 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:28:46.546527 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:28:46.546491 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:28:52.264897 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:28:52.264856 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:28:52.331031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:28:52.331006 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:28:52.402497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:28:52.402463 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:02.961020 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:02.960985 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:03.752027 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:03.751995 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:09.322138 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:09.322102 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:09.431612 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:09.431559 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:09.522815 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:09.522781 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:09.622862 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:09.622832 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:10.338731 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:10.338692 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:10.432335 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:10.432298 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:10.532532 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:10.532498 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:12.156781 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:12.156745 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:12.393631 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:12.393593 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:29:17.866878 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:17.866847 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:17.966991 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:17.966912 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:18.516645 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:18.516614 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:21.096921 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:21.096887 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:29:24.544188 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:24.544156 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:24.644223 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:24.644189 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:24.780629 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:24.780600 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:25.196232 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:25.196198 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:25.757802 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:25.757765 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:27.878578 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:27.878526 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:27.953973 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:27.953940 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:28.028554 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:28.028523 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:28.241920 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:28.241838 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:28.509123 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:28.509036 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:28.608944 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:28.608910 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:32.373176 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:32.373143 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:33.030184 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:33.030144 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:33.129705 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:33.129673 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:33.444777 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:33.444740 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:33.992796 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:33.992762 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:29:39.048959 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:39.048919 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:39.145305 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:39.145273 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:39.438747 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:39.438715 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:39.734055 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:39.733982 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:39.802259 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:39.802228 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:41.692185 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:41.692154 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:29:45.508646 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:45.508614 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:45.632697 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:45.632671 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:45.737414 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:45.737379 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:45.988648 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:45.988617 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:46.088334 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:46.088306 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:46.188458 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:46.188427 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:46.288797 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:46.288723 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:52.025423 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:52.025389 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:52.083109 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:52.083081 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:52.154072 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:52.154039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:52.259388 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:52.259364 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:52.371428 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:52.371395 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:57.946114 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:57.946045 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:29:58.047911 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:29:58.047874 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:04.658200 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:04.658155 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:04.990861 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:04.990771 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:05.257601 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:05.257494 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:08.070860 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:08.070821 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:08.152733 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:08.152697 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:08.210056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:08.209989 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:08.311143 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:08.311114 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:08.723599 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:08.723552 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:08.823443 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:08.823408 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:12.564807 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:12.564775 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:14.095580 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:14.095533 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:30:33.193208 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.193165 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv"] Jul 02 23:30:33.193767 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.193730 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed" containerName="authorino" Jul 02 23:30:33.193767 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.193751 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed" containerName="authorino" Jul 02 23:30:33.193882 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.193838 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fd4cb52-81a4-4e8e-a7e2-4107d482c5ed" containerName="authorino" Jul 02 23:30:33.196688 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.196665 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.200465 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.200438 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-de78a6e8\"/\"e2e-lifecycle-model-de78a6e8-kserve-self-signed-certs\"" Jul 02 23:30:33.200465 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.200449 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-de78a6e8\"/\"kube-root-ca.crt\"" Jul 02 23:30:33.200667 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.200539 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-de78a6e8\"/\"default-dockercfg-5glc5\"" Jul 02 23:30:33.200667 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.200542 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-de78a6e8\"/\"openshift-service-ca.crt\"" Jul 02 23:30:33.209056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.209034 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv"] Jul 02 23:30:33.225836 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.225808 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-dshm\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.225941 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.225846 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b2e378c-437b-4b9a-83b7-f799fc396599-tls-certs\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.225941 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.225929 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-model-cache\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.226017 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.225995 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch8pt\" (UniqueName: \"kubernetes.io/projected/4b2e378c-437b-4b9a-83b7-f799fc396599-kube-api-access-ch8pt\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.226056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.226019 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-home\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.226056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.226036 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-tmp-dir\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.289296 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.289269 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:33.326874 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.326851 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-home\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.326994 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.326881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-tmp-dir\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.326994 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.326918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-dshm\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.326994 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.326940 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b2e378c-437b-4b9a-83b7-f799fc396599-tls-certs\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.326994 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.326976 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-model-cache\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.327182 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.327029 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch8pt\" (UniqueName: \"kubernetes.io/projected/4b2e378c-437b-4b9a-83b7-f799fc396599-kube-api-access-ch8pt\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.327259 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.327242 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-tmp-dir\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.327320 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.327294 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-home\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.327407 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.327389 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-model-cache\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.329089 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.329070 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-dshm\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.329453 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.329437 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-de78a6e8\"/\"e2e-lifecycle-model-de78a6e8-kserve-self-signed-certs\"" Jul 02 23:30:33.334646 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.334627 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-de78a6e8\"/\"kube-root-ca.crt\"" Jul 02 23:30:33.339669 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.339648 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b2e378c-437b-4b9a-83b7-f799fc396599-tls-certs\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.345605 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.345561 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-de78a6e8\"/\"openshift-service-ca.crt\"" Jul 02 23:30:33.355636 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.355613 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch8pt\" (UniqueName: \"kubernetes.io/projected/4b2e378c-437b-4b9a-83b7-f799fc396599-kube-api-access-ch8pt\") pod \"e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.508106 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.508044 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-de78a6e8\"/\"default-dockercfg-5glc5\"" Jul 02 23:30:33.516118 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.516103 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:33.644962 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:33.644937 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv"] Jul 02 23:30:33.647550 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:30:33.647526 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b2e378c_437b_4b9a_83b7_f799fc396599.slice/crio-a5d681c886b8060915eeab073229df3fe2973a10b7800d838f819b946a2024e4 WatchSource:0}: Error finding container a5d681c886b8060915eeab073229df3fe2973a10b7800d838f819b946a2024e4: Status 404 returned error can't find the container with id a5d681c886b8060915eeab073229df3fe2973a10b7800d838f819b946a2024e4 Jul 02 23:30:34.068915 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:34.068879 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" event={"ID":"4b2e378c-437b-4b9a-83b7-f799fc396599","Type":"ContainerStarted","Data":"d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f"} Jul 02 23:30:34.068915 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:34.068921 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" event={"ID":"4b2e378c-437b-4b9a-83b7-f799fc396599","Type":"ContainerStarted","Data":"a5d681c886b8060915eeab073229df3fe2973a10b7800d838f819b946a2024e4"} Jul 02 23:30:34.085087 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:34.085042 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" podStartSLOduration=1.085028343 podStartE2EDuration="1.085028343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:30:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:30:34.084540676 +0000 UTC m=+1741.157793038" watchObservedRunningTime="2026-07-02 23:30:34.085028343 +0000 UTC m=+1741.158280732" Jul 02 23:30:36.391628 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:36.391592 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:30:39.036032 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:39.036000 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:39.086980 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:39.086940 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:43.516762 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:43.516732 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:43.517262 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:43.516772 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:43.529211 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:43.529190 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:44.132838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:44.132809 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:30:52.496647 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:52.496594 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:30:55.231187 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:55.231148 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:55.612214 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:55.612187 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:55.712747 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:55.712716 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:30:55.851838 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:30:55.851804 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:01.268738 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:01.268702 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:05.985793 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:05.985759 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:31:06.241288 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:06.241216 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv"] Jul 02 23:31:06.336837 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:06.336807 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:21.261270 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.261229 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" podUID="4b2e378c-437b-4b9a-83b7-f799fc396599" containerName="main" containerID="cri-o://d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f" gracePeriod=45 Jul 02 23:31:21.516217 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.516162 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:31:21.617424 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.617393 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch8pt\" (UniqueName: \"kubernetes.io/projected/4b2e378c-437b-4b9a-83b7-f799fc396599-kube-api-access-ch8pt\") pod \"4b2e378c-437b-4b9a-83b7-f799fc396599\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " Jul 02 23:31:21.617614 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.617433 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-home\") pod \"4b2e378c-437b-4b9a-83b7-f799fc396599\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " Jul 02 23:31:21.617614 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.617487 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-dshm\") pod \"4b2e378c-437b-4b9a-83b7-f799fc396599\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " Jul 02 23:31:21.617614 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.617520 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b2e378c-437b-4b9a-83b7-f799fc396599-tls-certs\") pod \"4b2e378c-437b-4b9a-83b7-f799fc396599\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " Jul 02 23:31:21.617614 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.617536 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-model-cache\") pod \"4b2e378c-437b-4b9a-83b7-f799fc396599\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " Jul 02 23:31:21.617614 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.617561 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-tmp-dir\") pod \"4b2e378c-437b-4b9a-83b7-f799fc396599\" (UID: \"4b2e378c-437b-4b9a-83b7-f799fc396599\") " Jul 02 23:31:21.617894 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.617795 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-home" (OuterVolumeSpecName: "home") pod "4b2e378c-437b-4b9a-83b7-f799fc396599" (UID: "4b2e378c-437b-4b9a-83b7-f799fc396599"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:31:21.617945 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.617886 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b2e378c-437b-4b9a-83b7-f799fc396599" (UID: "4b2e378c-437b-4b9a-83b7-f799fc396599"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:31:21.618017 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.617937 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b2e378c-437b-4b9a-83b7-f799fc396599" (UID: "4b2e378c-437b-4b9a-83b7-f799fc396599"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:31:21.619619 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.619593 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-dshm" (OuterVolumeSpecName: "dshm") pod "4b2e378c-437b-4b9a-83b7-f799fc396599" (UID: "4b2e378c-437b-4b9a-83b7-f799fc396599"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:31:21.619736 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.619642 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b2e378c-437b-4b9a-83b7-f799fc396599-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b2e378c-437b-4b9a-83b7-f799fc396599" (UID: "4b2e378c-437b-4b9a-83b7-f799fc396599"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:31:21.619736 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.619674 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b2e378c-437b-4b9a-83b7-f799fc396599-kube-api-access-ch8pt" (OuterVolumeSpecName: "kube-api-access-ch8pt") pod "4b2e378c-437b-4b9a-83b7-f799fc396599" (UID: "4b2e378c-437b-4b9a-83b7-f799fc396599"). InnerVolumeSpecName "kube-api-access-ch8pt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:31:21.718314 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.718288 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b2e378c-437b-4b9a-83b7-f799fc396599-tls-certs\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:31:21.718314 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.718315 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-model-cache\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:31:21.718480 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.718326 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-tmp-dir\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:31:21.718480 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.718338 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch8pt\" (UniqueName: \"kubernetes.io/projected/4b2e378c-437b-4b9a-83b7-f799fc396599-kube-api-access-ch8pt\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:31:21.718480 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.718350 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-home\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:31:21.718480 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:21.718361 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b2e378c-437b-4b9a-83b7-f799fc396599-dshm\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:31:22.306361 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:22.306323 2568 generic.go:356] "Generic (PLEG): container finished" podID="4b2e378c-437b-4b9a-83b7-f799fc396599" containerID="d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f" exitCode=0 Jul 02 23:31:22.306786 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:22.306368 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" Jul 02 23:31:22.306786 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:22.306410 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" event={"ID":"4b2e378c-437b-4b9a-83b7-f799fc396599","Type":"ContainerDied","Data":"d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f"} Jul 02 23:31:22.306786 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:22.306451 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv" event={"ID":"4b2e378c-437b-4b9a-83b7-f799fc396599","Type":"ContainerDied","Data":"a5d681c886b8060915eeab073229df3fe2973a10b7800d838f819b946a2024e4"} Jul 02 23:31:22.306786 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:22.306471 2568 scope.go:117] "RemoveContainer" containerID="d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f" Jul 02 23:31:22.315338 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:22.315321 2568 scope.go:117] "RemoveContainer" containerID="d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f" Jul 02 23:31:22.315589 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:31:22.315556 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f\": container with ID starting with d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f not found: ID does not exist" containerID="d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f" Jul 02 23:31:22.315657 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:22.315602 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f"} err="failed to get container status \"d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f\": rpc error: code = NotFound desc = could not find container \"d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f\": container with ID starting with d168946e713a2a9a0c3b833ff838a53fd003115dd9d0ba5181dc2b97926fcc8f not found: ID does not exist" Jul 02 23:31:22.328868 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:22.328847 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv"] Jul 02 23:31:22.333190 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:22.333171 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-de78a6e8/e2e-lifecycle-model-de78a6e8-kserve-6697bc6c7f-bv6sv"] Jul 02 23:31:23.468222 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:23.468187 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b2e378c-437b-4b9a-83b7-f799fc396599" path="/var/lib/kubelet/pods/4b2e378c-437b-4b9a-83b7-f799fc396599/volumes" Jul 02 23:31:33.521375 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:33.521344 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:31:33.527939 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:33.527919 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:31:42.517340 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:42.517305 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:43.830387 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:43.830351 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:45.400339 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:45.400307 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:31:48.625718 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:48.625654 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:54.200743 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:54.200683 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:54.255068 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:54.255039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:54.472870 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:54.472801 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:54.719892 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:54.719855 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:55.737005 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:55.736966 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:55.838401 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:55.838371 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:55.931789 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:55.931758 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:56.588253 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:56.588211 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:56.689580 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:56.689543 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:56.787588 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:56.787544 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:58.103479 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:58.103445 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:31:58.990670 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:31:58.990637 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:32:01.325278 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:01.325247 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:01.425847 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:01.425813 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:03.904490 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:03.904457 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:04.005323 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:04.005290 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:04.567871 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:04.567833 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:10.155114 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:10.155076 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:10.219911 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:10.219876 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:10.357452 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:10.357419 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:10.477234 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:10.477156 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:10.993607 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:10.993552 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:32:20.862871 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:20.862840 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:21.197475 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:21.197395 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:42.165825 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:42.165791 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:47.991032 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:47.990991 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:48.091306 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:48.091278 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:48.326333 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:48.326250 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:48.618255 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:48.618230 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:48.718507 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:48.718478 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:51.893367 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:51.893334 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:51.979243 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:51.979210 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:52.080758 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:52.080724 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:52.226814 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:52.226728 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:52.614237 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:52.614184 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:52.714990 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:52.714941 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:56.380799 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:56.380709 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:56.692432 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:56.692344 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:56.999308 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:56.999227 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:57.297617 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:57.297514 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:32:59.199804 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:59.199769 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:32:59.963961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:32:59.963925 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:00.099122 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:00.099078 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:00.281219 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:00.281143 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:00.551881 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:00.551809 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:00.751320 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:00.751281 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:00.848861 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:00.848779 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:04.403066 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:04.403029 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:10.094833 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:10.094798 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:33:13.295283 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:13.295249 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:13.395854 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:13.395823 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:22.323900 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:22.323871 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:22.423110 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:22.423081 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:35.976895 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:35.976856 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2"] Jul 02 23:33:35.977461 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:35.977370 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b2e378c-437b-4b9a-83b7-f799fc396599" containerName="main" Jul 02 23:33:35.977461 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:35.977390 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b2e378c-437b-4b9a-83b7-f799fc396599" containerName="main" Jul 02 23:33:35.977621 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:35.977474 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b2e378c-437b-4b9a-83b7-f799fc396599" containerName="main" Jul 02 23:33:35.980401 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:35.980380 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:35.983884 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:35.983850 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b0517f\"/\"test-model-b0517f-kserve-self-signed-certs\"" Jul 02 23:33:35.983884 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:35.983855 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b0517f\"/\"default-dockercfg-vr2wm\"" Jul 02 23:33:35.984057 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:35.983905 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b0517f\"/\"openshift-service-ca.crt\"" Jul 02 23:33:35.984057 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:35.983916 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b0517f\"/\"kube-root-ca.crt\"" Jul 02 23:33:35.988491 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:35.988367 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2"] Jul 02 23:33:36.079517 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.079486 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-home\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.079517 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.079519 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-model-cache\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.079700 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.079554 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tmp-dir\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.079700 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.079630 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tls-certs\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.079700 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.079665 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-dshm\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.079700 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.079682 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ctcf\" (UniqueName: \"kubernetes.io/projected/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-kube-api-access-5ctcf\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.091659 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.091609 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:33:36.180850 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.180824 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tmp-dir\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.180962 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.180857 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tls-certs\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.180962 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.180907 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-dshm\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.180962 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.180930 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ctcf\" (UniqueName: \"kubernetes.io/projected/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-kube-api-access-5ctcf\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.181126 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.181007 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-home\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.181126 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.181030 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-model-cache\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.181311 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.181277 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tmp-dir\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.181441 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.181385 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-home\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.181441 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.181398 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-model-cache\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.183178 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.183150 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-dshm\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.183419 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.183402 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tls-certs\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.188004 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.187984 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ctcf\" (UniqueName: \"kubernetes.io/projected/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-kube-api-access-5ctcf\") pod \"test-model-b0517f-kserve-55945dd7ff-dnbw2\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.290955 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.290891 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:36.442081 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.442056 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2"] Jul 02 23:33:36.443011 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:33:36.442985 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc73ba98e_3de1_4b1a_8882_1e5900e7ef87.slice/crio-be457b1ae6a9a62dd3e81cdd52be464c04f01175a1eb47b0bf8f7003631d4e40 WatchSource:0}: Error finding container be457b1ae6a9a62dd3e81cdd52be464c04f01175a1eb47b0bf8f7003631d4e40: Status 404 returned error can't find the container with id be457b1ae6a9a62dd3e81cdd52be464c04f01175a1eb47b0bf8f7003631d4e40 Jul 02 23:33:36.973330 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.973297 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" event={"ID":"c73ba98e-3de1-4b1a-8882-1e5900e7ef87","Type":"ContainerStarted","Data":"5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5"} Jul 02 23:33:36.973491 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.973342 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" event={"ID":"c73ba98e-3de1-4b1a-8882-1e5900e7ef87","Type":"ContainerStarted","Data":"be457b1ae6a9a62dd3e81cdd52be464c04f01175a1eb47b0bf8f7003631d4e40"} Jul 02 23:33:36.989196 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:36.989150 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" podStartSLOduration=1.989133829 podStartE2EDuration="1.989133829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:33:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:33:36.987694713 +0000 UTC m=+1924.060947099" watchObservedRunningTime="2026-07-02 23:33:36.989133829 +0000 UTC m=+1924.062386194" Jul 02 23:33:39.294514 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:39.294478 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:33:46.291008 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:46.290974 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:46.291400 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:46.291230 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:46.303502 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:46.303480 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:47.035392 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:47.035365 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:33:48.991585 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:33:48.991532 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:34:11.177804 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.177772 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:34:11.201714 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.199883 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw"] Jul 02 23:34:11.204685 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.204650 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.207349 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.207327 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0ae6cc\"/\"kube-root-ca.crt\"" Jul 02 23:34:11.208635 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.208612 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0ae6cc\"/\"openshift-service-ca.crt\"" Jul 02 23:34:11.208759 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.208735 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0ae6cc\"/\"test-model-0ae6cc-kserve-self-signed-certs\"" Jul 02 23:34:11.208997 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.208868 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0ae6cc\"/\"default-dockercfg-x49wl\"" Jul 02 23:34:11.211229 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.211207 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw"] Jul 02 23:34:11.235791 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.235767 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:34:11.263332 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.263311 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-model-cache\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.263447 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.263341 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-dshm\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.263447 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.263378 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-home\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.263447 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.263429 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tmp-dir\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.263614 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.263465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj2kw\" (UniqueName: \"kubernetes.io/projected/71bc060c-d5a8-4a53-88ea-f97514d26ccd-kube-api-access-hj2kw\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.263614 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.263484 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tls-certs\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.363920 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.363893 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tmp-dir\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.364021 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.363938 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj2kw\" (UniqueName: \"kubernetes.io/projected/71bc060c-d5a8-4a53-88ea-f97514d26ccd-kube-api-access-hj2kw\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.364021 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.363963 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tls-certs\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.364021 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.363995 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-model-cache\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.364021 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.364014 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-dshm\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.364192 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.364049 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-home\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.364331 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.364310 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tmp-dir\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.364374 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.364336 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-model-cache\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.364457 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.364440 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-home\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.366330 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.366309 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-dshm\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.366414 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.366387 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tls-certs\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.370981 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.370958 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj2kw\" (UniqueName: \"kubernetes.io/projected/71bc060c-d5a8-4a53-88ea-f97514d26ccd-kube-api-access-hj2kw\") pod \"test-model-0ae6cc-kserve-8498bc6f8f-t2rlw\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.515813 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.515756 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:11.641143 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:11.641118 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw"] Jul 02 23:34:11.643069 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:34:11.643029 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71bc060c_d5a8_4a53_88ea_f97514d26ccd.slice/crio-d67f2f7220d048050621c387e7b3b0ddcc03ceb41b94715478685a89be1d511f WatchSource:0}: Error finding container d67f2f7220d048050621c387e7b3b0ddcc03ceb41b94715478685a89be1d511f: Status 404 returned error can't find the container with id d67f2f7220d048050621c387e7b3b0ddcc03ceb41b94715478685a89be1d511f Jul 02 23:34:12.157419 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:12.157378 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" event={"ID":"71bc060c-d5a8-4a53-88ea-f97514d26ccd","Type":"ContainerStarted","Data":"10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3"} Jul 02 23:34:12.157419 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:12.157423 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" event={"ID":"71bc060c-d5a8-4a53-88ea-f97514d26ccd","Type":"ContainerStarted","Data":"d67f2f7220d048050621c387e7b3b0ddcc03ceb41b94715478685a89be1d511f"} Jul 02 23:34:12.176067 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:12.176016 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" podStartSLOduration=1.176001632 podStartE2EDuration="1.176001632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:34:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:34:12.174088816 +0000 UTC m=+1959.247341181" watchObservedRunningTime="2026-07-02 23:34:12.176001632 +0000 UTC m=+1959.249254039" Jul 02 23:34:14.994479 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:14.994439 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:34:21.516523 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:21.516492 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:21.516901 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:21.516538 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:21.528721 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:21.528698 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:22.217854 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:22.217823 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:34.926913 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:34.926880 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:34:38.582897 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:38.582865 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2"] Jul 02 23:34:38.599499 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:38.599471 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:34:39.486306 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:39.486271 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw"] Jul 02 23:34:39.504699 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:39.504672 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:34:39.679588 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:39.679540 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:34:40.053125 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:40.053096 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:34:40.158679 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:40.158655 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:34:40.300072 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:40.300046 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:34:41.992439 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:41.992404 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:34:45.129028 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:45.128992 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:34:53.603904 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.603840 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" podUID="c73ba98e-3de1-4b1a-8882-1e5900e7ef87" containerName="main" containerID="cri-o://5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5" gracePeriod=45 Jul 02 23:34:53.853034 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.853006 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:34:53.900050 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900021 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-home\") pod \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " Jul 02 23:34:53.900182 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900068 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tmp-dir\") pod \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " Jul 02 23:34:53.900182 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900085 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-model-cache\") pod \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " Jul 02 23:34:53.900182 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900126 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tls-certs\") pod \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " Jul 02 23:34:53.900182 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900156 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-dshm\") pod \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " Jul 02 23:34:53.900342 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900192 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ctcf\" (UniqueName: \"kubernetes.io/projected/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-kube-api-access-5ctcf\") pod \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\" (UID: \"c73ba98e-3de1-4b1a-8882-1e5900e7ef87\") " Jul 02 23:34:53.900342 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900284 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-home" (OuterVolumeSpecName: "home") pod "c73ba98e-3de1-4b1a-8882-1e5900e7ef87" (UID: "c73ba98e-3de1-4b1a-8882-1e5900e7ef87"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:53.900342 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900303 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c73ba98e-3de1-4b1a-8882-1e5900e7ef87" (UID: "c73ba98e-3de1-4b1a-8882-1e5900e7ef87"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:53.900482 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900399 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-model-cache" (OuterVolumeSpecName: "model-cache") pod "c73ba98e-3de1-4b1a-8882-1e5900e7ef87" (UID: "c73ba98e-3de1-4b1a-8882-1e5900e7ef87"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:53.900482 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900460 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-home\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:53.900482 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900471 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tmp-dir\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:53.900482 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.900480 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-model-cache\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:53.902323 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.902295 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c73ba98e-3de1-4b1a-8882-1e5900e7ef87" (UID: "c73ba98e-3de1-4b1a-8882-1e5900e7ef87"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:34:53.902441 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.902389 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-dshm" (OuterVolumeSpecName: "dshm") pod "c73ba98e-3de1-4b1a-8882-1e5900e7ef87" (UID: "c73ba98e-3de1-4b1a-8882-1e5900e7ef87"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:53.902497 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:53.902436 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-kube-api-access-5ctcf" (OuterVolumeSpecName: "kube-api-access-5ctcf") pod "c73ba98e-3de1-4b1a-8882-1e5900e7ef87" (UID: "c73ba98e-3de1-4b1a-8882-1e5900e7ef87"). InnerVolumeSpecName "kube-api-access-5ctcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:34:54.001783 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.001760 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-tls-certs\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:54.001783 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.001783 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-dshm\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:54.001941 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.001797 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ctcf\" (UniqueName: \"kubernetes.io/projected/c73ba98e-3de1-4b1a-8882-1e5900e7ef87-kube-api-access-5ctcf\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:54.364684 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.364650 2568 generic.go:356] "Generic (PLEG): container finished" podID="c73ba98e-3de1-4b1a-8882-1e5900e7ef87" containerID="5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5" exitCode=0 Jul 02 23:34:54.364833 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.364693 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" Jul 02 23:34:54.364833 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.364693 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" event={"ID":"c73ba98e-3de1-4b1a-8882-1e5900e7ef87","Type":"ContainerDied","Data":"5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5"} Jul 02 23:34:54.364833 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.364791 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2" event={"ID":"c73ba98e-3de1-4b1a-8882-1e5900e7ef87","Type":"ContainerDied","Data":"be457b1ae6a9a62dd3e81cdd52be464c04f01175a1eb47b0bf8f7003631d4e40"} Jul 02 23:34:54.364833 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.364813 2568 scope.go:117] "RemoveContainer" containerID="5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5" Jul 02 23:34:54.373879 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.373858 2568 scope.go:117] "RemoveContainer" containerID="5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5" Jul 02 23:34:54.374116 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:34:54.374097 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5\": container with ID starting with 5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5 not found: ID does not exist" containerID="5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5" Jul 02 23:34:54.374176 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.374124 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5"} err="failed to get container status \"5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5\": rpc error: code = NotFound desc = could not find container \"5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5\": container with ID starting with 5fbce4c1960643450288f06b5da1812b52b8b3bca950c2f767a0fc296b714ae5 not found: ID does not exist" Jul 02 23:34:54.386029 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.386005 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2"] Jul 02 23:34:54.388474 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.388452 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-b0517f/test-model-b0517f-kserve-55945dd7ff-dnbw2"] Jul 02 23:34:54.507677 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.507634 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" podUID="71bc060c-d5a8-4a53-88ea-f97514d26ccd" containerName="main" containerID="cri-o://10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3" gracePeriod=45 Jul 02 23:34:54.689433 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.689399 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:34:54.750300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.750279 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:54.808035 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.808006 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-dshm\") pod \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " Jul 02 23:34:54.808164 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.808056 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj2kw\" (UniqueName: \"kubernetes.io/projected/71bc060c-d5a8-4a53-88ea-f97514d26ccd-kube-api-access-hj2kw\") pod \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " Jul 02 23:34:54.808164 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.808097 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tls-certs\") pod \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " Jul 02 23:34:54.808164 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.808140 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-model-cache\") pod \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " Jul 02 23:34:54.808164 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.808156 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tmp-dir\") pod \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " Jul 02 23:34:54.808379 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.808189 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-home\") pod \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\" (UID: \"71bc060c-d5a8-4a53-88ea-f97514d26ccd\") " Jul 02 23:34:54.808453 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.808406 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "71bc060c-d5a8-4a53-88ea-f97514d26ccd" (UID: "71bc060c-d5a8-4a53-88ea-f97514d26ccd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:54.808519 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.808418 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-model-cache" (OuterVolumeSpecName: "model-cache") pod "71bc060c-d5a8-4a53-88ea-f97514d26ccd" (UID: "71bc060c-d5a8-4a53-88ea-f97514d26ccd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:54.808519 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.808481 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-home" (OuterVolumeSpecName: "home") pod "71bc060c-d5a8-4a53-88ea-f97514d26ccd" (UID: "71bc060c-d5a8-4a53-88ea-f97514d26ccd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:54.810227 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.810200 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "71bc060c-d5a8-4a53-88ea-f97514d26ccd" (UID: "71bc060c-d5a8-4a53-88ea-f97514d26ccd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:34:54.810227 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.810214 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-dshm" (OuterVolumeSpecName: "dshm") pod "71bc060c-d5a8-4a53-88ea-f97514d26ccd" (UID: "71bc060c-d5a8-4a53-88ea-f97514d26ccd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:34:54.810360 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.810276 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71bc060c-d5a8-4a53-88ea-f97514d26ccd-kube-api-access-hj2kw" (OuterVolumeSpecName: "kube-api-access-hj2kw") pod "71bc060c-d5a8-4a53-88ea-f97514d26ccd" (UID: "71bc060c-d5a8-4a53-88ea-f97514d26ccd"). InnerVolumeSpecName "kube-api-access-hj2kw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:34:54.909300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.909243 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-home\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:54.909300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.909264 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-dshm\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:54.909300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.909274 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj2kw\" (UniqueName: \"kubernetes.io/projected/71bc060c-d5a8-4a53-88ea-f97514d26ccd-kube-api-access-hj2kw\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:54.909300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.909285 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tls-certs\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:54.909300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.909297 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-model-cache\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:54.909525 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:54.909309 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71bc060c-d5a8-4a53-88ea-f97514d26ccd-tmp-dir\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:34:55.371993 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:55.371956 2568 generic.go:356] "Generic (PLEG): container finished" podID="71bc060c-d5a8-4a53-88ea-f97514d26ccd" containerID="10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3" exitCode=0 Jul 02 23:34:55.372148 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:55.372001 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" Jul 02 23:34:55.372148 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:55.372048 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" event={"ID":"71bc060c-d5a8-4a53-88ea-f97514d26ccd","Type":"ContainerDied","Data":"10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3"} Jul 02 23:34:55.372148 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:55.372085 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw" event={"ID":"71bc060c-d5a8-4a53-88ea-f97514d26ccd","Type":"ContainerDied","Data":"d67f2f7220d048050621c387e7b3b0ddcc03ceb41b94715478685a89be1d511f"} Jul 02 23:34:55.372148 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:55.372105 2568 scope.go:117] "RemoveContainer" containerID="10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3" Jul 02 23:34:55.381674 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:55.381657 2568 scope.go:117] "RemoveContainer" containerID="10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3" Jul 02 23:34:55.381943 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:34:55.381926 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3\": container with ID starting with 10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3 not found: ID does not exist" containerID="10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3" Jul 02 23:34:55.381992 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:55.381951 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3"} err="failed to get container status \"10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3\": rpc error: code = NotFound desc = could not find container \"10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3\": container with ID starting with 10f6718fe2abdbfe1631b28e96fa5654c27e2a6ed690a1a0bcce9f004d3833f3 not found: ID does not exist" Jul 02 23:34:55.394179 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:55.394153 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw"] Jul 02 23:34:55.396183 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:55.396158 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0ae6cc/test-model-0ae6cc-kserve-8498bc6f8f-t2rlw"] Jul 02 23:34:55.469857 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:55.469831 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71bc060c-d5a8-4a53-88ea-f97514d26ccd" path="/var/lib/kubelet/pods/71bc060c-d5a8-4a53-88ea-f97514d26ccd/volumes" Jul 02 23:34:55.470220 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:34:55.470207 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c73ba98e-3de1-4b1a-8882-1e5900e7ef87" path="/var/lib/kubelet/pods/c73ba98e-3de1-4b1a-8882-1e5900e7ef87/volumes" Jul 02 23:35:21.685146 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:21.685105 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:21.939374 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:21.939299 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:22.176296 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:22.176263 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:22.341189 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:22.341099 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:22.727483 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:22.727451 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:22.828914 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:22.828885 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:22.972687 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:22.972653 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:25.733443 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:25.733408 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:35:28.512721 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:28.512688 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:28.612395 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:28.612361 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:28.770778 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:28.770692 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:28.989465 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:28.989434 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:29.317386 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:29.317352 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:29.623637 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:29.623608 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:29.747426 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:29.747379 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:32.727156 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:32.727118 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:32.811397 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:32.811363 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:32.878100 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:32.878064 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:32.978354 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:32.978279 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:33.287013 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:33.286932 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:33.390165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:33.390138 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:37.223542 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:37.223508 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:37.591639 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:37.591547 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:35:37.663775 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:37.663740 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:37.960449 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:37.960419 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:38.359944 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:38.359868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:39.751413 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:39.751383 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:39.851504 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:39.851473 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:40.850258 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:40.850214 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:40.923619 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:40.923578 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:40.990561 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:40.990533 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:41.294466 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:41.294437 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:41.515792 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:41.515759 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:41.604830 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:41.604747 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:45.238738 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:45.238697 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:47.192295 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:47.192255 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:35:53.250618 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:53.250550 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:53.610999 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:53.610918 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:53.715978 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:53.715940 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:53.812889 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:53.812858 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:53.929937 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:53.929898 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:57.391315 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:57.391239 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:35:59.379610 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:59.379543 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:59.479590 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:59.479542 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:59.632519 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:59.632440 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:59.845045 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:59.845009 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:35:59.898094 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:35:59.898023 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:00.015537 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:00.015505 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:00.384658 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:00.384624 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:00.485031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:00.484997 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:00.585451 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:00.585420 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:00.773670 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:00.773590 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:06.063350 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:06.063300 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:06.164463 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:06.164433 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:06.320583 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:06.320484 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:06.535505 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:06.535474 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:06.854882 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:06.854848 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:07.149095 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:07.149060 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:07.384216 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:07.384183 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:09.694292 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:09.694258 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:36:09.921455 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:09.921418 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:10.009703 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:10.009621 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:10.093378 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:10.093342 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:10.215366 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:10.215334 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:10.485490 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:10.485454 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:10.582629 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:10.582598 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:14.407428 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:14.407393 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:14.725947 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:14.725864 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:15.024254 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:15.024179 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:15.304147 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:15.304072 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:18.023993 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:18.023915 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:18.124020 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:18.123985 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:18.222629 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:18.222597 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:18.571110 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:18.571077 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:18.770621 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:18.770585 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:18.894059 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:18.894025 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:20.393371 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:20.393330 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:36:22.485036 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:22.485006 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:31.185582 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:31.185528 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:31.286415 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:31.286385 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:32.588650 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.588616 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj"] Jul 02 23:36:32.589114 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.588933 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71bc060c-d5a8-4a53-88ea-f97514d26ccd" containerName="main" Jul 02 23:36:32.589114 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.588943 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="71bc060c-d5a8-4a53-88ea-f97514d26ccd" containerName="main" Jul 02 23:36:32.589114 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.588960 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c73ba98e-3de1-4b1a-8882-1e5900e7ef87" containerName="main" Jul 02 23:36:32.589114 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.588965 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c73ba98e-3de1-4b1a-8882-1e5900e7ef87" containerName="main" Jul 02 23:36:32.589114 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.589025 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c73ba98e-3de1-4b1a-8882-1e5900e7ef87" containerName="main" Jul 02 23:36:32.589114 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.589032 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="71bc060c-d5a8-4a53-88ea-f97514d26ccd" containerName="main" Jul 02 23:36:32.592116 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.592096 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.594614 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.594589 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9352c3\"/\"default-dockercfg-zb6ps\"" Jul 02 23:36:32.594614 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.594604 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9352c3\"/\"openshift-service-ca.crt\"" Jul 02 23:36:32.595722 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.595701 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9352c3\"/\"auth-test-model-9352c3-kserve-self-signed-certs\"" Jul 02 23:36:32.595822 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.595745 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9352c3\"/\"kube-root-ca.crt\"" Jul 02 23:36:32.601032 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.601009 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj"] Jul 02 23:36:32.619031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.619009 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:32.708124 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.708095 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tls-certs\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.708239 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.708145 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-model-cache\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.708239 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.708179 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4s48\" (UniqueName: \"kubernetes.io/projected/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-kube-api-access-s4s48\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.708239 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.708203 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-home\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.708342 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.708251 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-dshm\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.708342 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.708291 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tmp-dir\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.808660 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.808633 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tmp-dir\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.808790 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.808667 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tls-certs\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.808790 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.808708 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-model-cache\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.808790 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.808738 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4s48\" (UniqueName: \"kubernetes.io/projected/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-kube-api-access-s4s48\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.808790 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.808757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-home\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.808994 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.808805 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-dshm\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.809113 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.809087 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tmp-dir\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.809214 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.809110 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-model-cache\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.809267 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.809214 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-home\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.810924 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.810901 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-dshm\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.811209 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.811188 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tls-certs\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.816971 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.816950 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4s48\" (UniqueName: \"kubernetes.io/projected/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-kube-api-access-s4s48\") pod \"auth-test-model-9352c3-kserve-5db896875f-wpdrj\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:32.902410 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:32.902383 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:33.035282 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:33.035242 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj"] Jul 02 23:36:33.036723 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:36:33.036687 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2b9a6fe_877a_4b14_8470_2b0ce21a54e4.slice/crio-c29d15881d6a0bf61bbe294350f36d43bfe682eae6fdef0c10d5dfb895ab08b8 WatchSource:0}: Error finding container c29d15881d6a0bf61bbe294350f36d43bfe682eae6fdef0c10d5dfb895ab08b8: Status 404 returned error can't find the container with id c29d15881d6a0bf61bbe294350f36d43bfe682eae6fdef0c10d5dfb895ab08b8 Jul 02 23:36:33.558603 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:33.558553 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:36:33.566301 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:33.566270 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:36:33.876706 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:33.876675 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" event={"ID":"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4","Type":"ContainerStarted","Data":"c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d"} Jul 02 23:36:33.877038 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:33.876717 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" event={"ID":"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4","Type":"ContainerStarted","Data":"c29d15881d6a0bf61bbe294350f36d43bfe682eae6fdef0c10d5dfb895ab08b8"} Jul 02 23:36:33.894009 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:33.893965 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" podStartSLOduration=1.8939514979999998 podStartE2EDuration="1.893951498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:36:33.892134855 +0000 UTC m=+2100.965387221" watchObservedRunningTime="2026-07-02 23:36:33.893951498 +0000 UTC m=+2100.967203838" Jul 02 23:36:36.088357 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:36.088321 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:36:40.336363 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:40.336327 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:40.435270 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:40.435235 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:36:42.902822 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:42.902779 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:42.902822 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:42.902828 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:42.915115 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:42.915083 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:36:42.933475 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:36:42.933450 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:37:34.693396 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.693351 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n"] Jul 02 23:37:34.696816 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.696798 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.699106 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.699081 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"openshift-service-ca.crt\"" Jul 02 23:37:34.699212 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.699120 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"kube-root-ca.crt\"" Jul 02 23:37:34.700118 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.700098 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"default-dockercfg-8qfd8\"" Jul 02 23:37:34.700215 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.700164 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"auth-test-model-83f2de-kserve-self-signed-certs\"" Jul 02 23:37:34.713173 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.708161 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n"] Jul 02 23:37:34.727110 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.727088 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:37:34.816220 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.816195 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddc034c0-93d8-4bfc-bd0b-878660843808-tls-certs\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.816357 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.816226 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrs9d\" (UniqueName: \"kubernetes.io/projected/ddc034c0-93d8-4bfc-bd0b-878660843808-kube-api-access-hrs9d\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.816357 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.816338 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-home\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.816432 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.816379 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-tmp-dir\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.816432 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.816406 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-dshm\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.816502 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.816455 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-model-cache\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.917876 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.917844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-home\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.917876 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.917881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-tmp-dir\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.918108 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.917898 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-dshm\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.918108 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.917927 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-model-cache\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.918108 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.917952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddc034c0-93d8-4bfc-bd0b-878660843808-tls-certs\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.918108 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.917969 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrs9d\" (UniqueName: \"kubernetes.io/projected/ddc034c0-93d8-4bfc-bd0b-878660843808-kube-api-access-hrs9d\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.918335 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.918309 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-home\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.918468 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.918350 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-tmp-dir\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.918468 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.918426 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-model-cache\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.920113 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.920095 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-dshm\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.920389 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.920369 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddc034c0-93d8-4bfc-bd0b-878660843808-tls-certs\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:34.926473 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:34.926441 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrs9d\" (UniqueName: \"kubernetes.io/projected/ddc034c0-93d8-4bfc-bd0b-878660843808-kube-api-access-hrs9d\") pod \"auth-test-model-83f2de-kserve-668fd74db7-lgn4n\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:35.007401 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:35.007345 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:35.132781 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:35.132749 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n"] Jul 02 23:37:35.134061 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:37:35.134028 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddc034c0_93d8_4bfc_bd0b_878660843808.slice/crio-91686f384840b75d66ddab48379d1aa54651bfc92956bd209e1f02010d48559c WatchSource:0}: Error finding container 91686f384840b75d66ddab48379d1aa54651bfc92956bd209e1f02010d48559c: Status 404 returned error can't find the container with id 91686f384840b75d66ddab48379d1aa54651bfc92956bd209e1f02010d48559c Jul 02 23:37:35.184795 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:35.184729 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" event={"ID":"ddc034c0-93d8-4bfc-bd0b-878660843808","Type":"ContainerStarted","Data":"91686f384840b75d66ddab48379d1aa54651bfc92956bd209e1f02010d48559c"} Jul 02 23:37:36.191259 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:36.191218 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" event={"ID":"ddc034c0-93d8-4bfc-bd0b-878660843808","Type":"ContainerStarted","Data":"9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2"} Jul 02 23:37:36.207890 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:36.207842 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" podStartSLOduration=2.207827132 podStartE2EDuration="2.207827132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:37:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:37:36.207143007 +0000 UTC m=+2163.280395370" watchObservedRunningTime="2026-07-02 23:37:36.207827132 +0000 UTC m=+2163.281079509" Jul 02 23:37:38.187231 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:38.187198 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:37:45.007710 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:45.007678 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:45.007710 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:45.007722 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:45.020523 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:45.020495 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:45.245832 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:45.245801 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:37:51.499165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:37:51.499121 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:38:09.419339 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:38:09.419306 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:38:23.291499 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:38:23.291458 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:38:36.893732 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:38:36.893695 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:38:38.396795 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:38:38.396760 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:38:54.987320 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:38:54.987287 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:39:09.683344 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:09.683264 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:39:22.474391 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:22.474355 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj"] Jul 02 23:39:22.490422 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:22.490393 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:39:23.421116 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:23.421080 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n"] Jul 02 23:39:23.431973 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:23.431946 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:39:25.988582 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:25.988545 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:39:28.199962 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.199929 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv"] Jul 02 23:39:28.203478 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.203461 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.206149 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.206126 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9352c3\"/\"sub-test-model-9352c3-kserve-self-signed-certs\"" Jul 02 23:39:28.211799 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.211779 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv"] Jul 02 23:39:28.215672 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.215649 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:39:28.346768 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.346736 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-model-cache\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.346935 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.346807 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-dshm\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.346935 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.346845 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tmp-dir\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.346935 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.346870 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vj7g\" (UniqueName: \"kubernetes.io/projected/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-kube-api-access-4vj7g\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.347052 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.346954 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-home\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.347052 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.346990 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tls-certs\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.447920 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.447885 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-model-cache\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.448066 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.447975 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-dshm\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.448066 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.448026 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tmp-dir\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.448184 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.448055 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vj7g\" (UniqueName: \"kubernetes.io/projected/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-kube-api-access-4vj7g\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.448184 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.448113 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-home\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.448184 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.448148 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tls-certs\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.451361 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.451295 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tls-certs\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.451903 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.451617 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-model-cache\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.454646 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.454001 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-dshm\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.454646 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.454252 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tmp-dir\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.454804 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.454685 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-home\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.463074 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.463050 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vj7g\" (UniqueName: \"kubernetes.io/projected/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-kube-api-access-4vj7g\") pod \"sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.513194 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.513170 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:28.642977 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.642946 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv"] Jul 02 23:39:28.644444 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:39:28.644422 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0743a3e7_6a3a_4cbb_8e9b_4400d90174d9.slice/crio-7f6f38caa2862303ab1ff1a0848f88d7d40d1aa39e28686f1de03ddf02e25bea WatchSource:0}: Error finding container 7f6f38caa2862303ab1ff1a0848f88d7d40d1aa39e28686f1de03ddf02e25bea: Status 404 returned error can't find the container with id 7f6f38caa2862303ab1ff1a0848f88d7d40d1aa39e28686f1de03ddf02e25bea Jul 02 23:39:28.754684 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.754603 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" event={"ID":"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9","Type":"ContainerStarted","Data":"cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf"} Jul 02 23:39:28.754684 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.754646 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" event={"ID":"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9","Type":"ContainerStarted","Data":"7f6f38caa2862303ab1ff1a0848f88d7d40d1aa39e28686f1de03ddf02e25bea"} Jul 02 23:39:28.773464 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:28.773411 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" podStartSLOduration=0.773396937 podStartE2EDuration="773.396937ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:39:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:39:28.771308938 +0000 UTC m=+2275.844561302" watchObservedRunningTime="2026-07-02 23:39:28.773396937 +0000 UTC m=+2275.846649300" Jul 02 23:39:37.494313 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.494255 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" podUID="a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" containerName="main" containerID="cri-o://c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d" gracePeriod=45 Jul 02 23:39:37.746815 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.746755 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:39:37.799477 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.799448 2568 generic.go:356] "Generic (PLEG): container finished" podID="a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" containerID="c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d" exitCode=0 Jul 02 23:39:37.799643 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.799490 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" Jul 02 23:39:37.799643 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.799538 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" event={"ID":"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4","Type":"ContainerDied","Data":"c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d"} Jul 02 23:39:37.799643 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.799597 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj" event={"ID":"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4","Type":"ContainerDied","Data":"c29d15881d6a0bf61bbe294350f36d43bfe682eae6fdef0c10d5dfb895ab08b8"} Jul 02 23:39:37.799643 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.799618 2568 scope.go:117] "RemoveContainer" containerID="c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d" Jul 02 23:39:37.808080 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.808064 2568 scope.go:117] "RemoveContainer" containerID="c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d" Jul 02 23:39:37.808298 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:39:37.808282 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d\": container with ID starting with c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d not found: ID does not exist" containerID="c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d" Jul 02 23:39:37.808344 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.808308 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d"} err="failed to get container status \"c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d\": rpc error: code = NotFound desc = could not find container \"c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d\": container with ID starting with c031129b086c712cfbb6564087efceeb15d833f193c5b92b4d67ba20f0a6af5d not found: ID does not exist" Jul 02 23:39:37.831146 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.831123 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4s48\" (UniqueName: \"kubernetes.io/projected/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-kube-api-access-s4s48\") pod \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " Jul 02 23:39:37.831236 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.831226 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tls-certs\") pod \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " Jul 02 23:39:37.831289 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.831253 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-dshm\") pod \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " Jul 02 23:39:37.831289 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.831275 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-model-cache\") pod \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " Jul 02 23:39:37.831395 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.831290 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-home\") pod \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " Jul 02 23:39:37.831395 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.831305 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tmp-dir\") pod \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\" (UID: \"a2b9a6fe-877a-4b14-8470-2b0ce21a54e4\") " Jul 02 23:39:37.831546 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.831514 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" (UID: "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:37.831687 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.831599 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-home" (OuterVolumeSpecName: "home") pod "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" (UID: "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:37.831687 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.831627 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" (UID: "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:37.833367 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.833348 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" (UID: "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:39:37.833436 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.833399 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-dshm" (OuterVolumeSpecName: "dshm") pod "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" (UID: "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:37.833480 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.833457 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-kube-api-access-s4s48" (OuterVolumeSpecName: "kube-api-access-s4s48") pod "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" (UID: "a2b9a6fe-877a-4b14-8470-2b0ce21a54e4"). InnerVolumeSpecName "kube-api-access-s4s48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:39:37.932582 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.932541 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tls-certs\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:37.932582 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.932583 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-dshm\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:37.932727 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.932592 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-model-cache\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:37.932727 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.932599 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-home\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:37.932727 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.932607 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-tmp-dir\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:37.932727 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:37.932615 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4s48\" (UniqueName: \"kubernetes.io/projected/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4-kube-api-access-s4s48\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:38.121692 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.121666 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj"] Jul 02 23:39:38.123655 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.123634 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/auth-test-model-9352c3-kserve-5db896875f-wpdrj"] Jul 02 23:39:38.440625 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.440590 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" podUID="ddc034c0-93d8-4bfc-bd0b-878660843808" containerName="main" containerID="cri-o://9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2" gracePeriod=45 Jul 02 23:39:38.514100 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.514065 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:38.514100 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.514116 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:38.526633 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.526610 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:38.692976 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.692923 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:39:38.806027 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.805992 2568 generic.go:356] "Generic (PLEG): container finished" podID="ddc034c0-93d8-4bfc-bd0b-878660843808" containerID="9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2" exitCode=0 Jul 02 23:39:38.806204 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.806048 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" Jul 02 23:39:38.806204 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.806082 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" event={"ID":"ddc034c0-93d8-4bfc-bd0b-878660843808","Type":"ContainerDied","Data":"9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2"} Jul 02 23:39:38.806204 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.806133 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n" event={"ID":"ddc034c0-93d8-4bfc-bd0b-878660843808","Type":"ContainerDied","Data":"91686f384840b75d66ddab48379d1aa54651bfc92956bd209e1f02010d48559c"} Jul 02 23:39:38.806204 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.806150 2568 scope.go:117] "RemoveContainer" containerID="9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2" Jul 02 23:39:38.814761 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.814734 2568 scope.go:117] "RemoveContainer" containerID="9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2" Jul 02 23:39:38.815044 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:39:38.815026 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2\": container with ID starting with 9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2 not found: ID does not exist" containerID="9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2" Jul 02 23:39:38.815108 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.815051 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2"} err="failed to get container status \"9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2\": rpc error: code = NotFound desc = could not find container \"9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2\": container with ID starting with 9f60960ccb90c4d36832abfed9d730d77419a45cbdfac7d3f2a90f02b76f86e2 not found: ID does not exist" Jul 02 23:39:38.818782 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.818756 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:39:38.839253 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.839228 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddc034c0-93d8-4bfc-bd0b-878660843808-tls-certs\") pod \"ddc034c0-93d8-4bfc-bd0b-878660843808\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " Jul 02 23:39:38.839374 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.839259 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-model-cache\") pod \"ddc034c0-93d8-4bfc-bd0b-878660843808\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " Jul 02 23:39:38.839374 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.839307 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-home\") pod \"ddc034c0-93d8-4bfc-bd0b-878660843808\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " Jul 02 23:39:38.839374 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.839349 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrs9d\" (UniqueName: \"kubernetes.io/projected/ddc034c0-93d8-4bfc-bd0b-878660843808-kube-api-access-hrs9d\") pod \"ddc034c0-93d8-4bfc-bd0b-878660843808\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " Jul 02 23:39:38.839374 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.839371 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-dshm\") pod \"ddc034c0-93d8-4bfc-bd0b-878660843808\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " Jul 02 23:39:38.839624 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.839437 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-tmp-dir\") pod \"ddc034c0-93d8-4bfc-bd0b-878660843808\" (UID: \"ddc034c0-93d8-4bfc-bd0b-878660843808\") " Jul 02 23:39:38.839703 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.839629 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-home" (OuterVolumeSpecName: "home") pod "ddc034c0-93d8-4bfc-bd0b-878660843808" (UID: "ddc034c0-93d8-4bfc-bd0b-878660843808"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:38.839783 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.839758 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-model-cache" (OuterVolumeSpecName: "model-cache") pod "ddc034c0-93d8-4bfc-bd0b-878660843808" (UID: "ddc034c0-93d8-4bfc-bd0b-878660843808"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:38.839844 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.839804 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ddc034c0-93d8-4bfc-bd0b-878660843808" (UID: "ddc034c0-93d8-4bfc-bd0b-878660843808"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:38.841475 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.841453 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddc034c0-93d8-4bfc-bd0b-878660843808-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ddc034c0-93d8-4bfc-bd0b-878660843808" (UID: "ddc034c0-93d8-4bfc-bd0b-878660843808"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:39:38.841583 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.841482 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-dshm" (OuterVolumeSpecName: "dshm") pod "ddc034c0-93d8-4bfc-bd0b-878660843808" (UID: "ddc034c0-93d8-4bfc-bd0b-878660843808"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:39:38.841583 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.841534 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc034c0-93d8-4bfc-bd0b-878660843808-kube-api-access-hrs9d" (OuterVolumeSpecName: "kube-api-access-hrs9d") pod "ddc034c0-93d8-4bfc-bd0b-878660843808" (UID: "ddc034c0-93d8-4bfc-bd0b-878660843808"). InnerVolumeSpecName "kube-api-access-hrs9d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:39:38.940306 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.940272 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrs9d\" (UniqueName: \"kubernetes.io/projected/ddc034c0-93d8-4bfc-bd0b-878660843808-kube-api-access-hrs9d\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:38.940306 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.940299 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-dshm\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:38.940306 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.940311 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-tmp-dir\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:38.940616 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.940320 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddc034c0-93d8-4bfc-bd0b-878660843808-tls-certs\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:38.940616 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.940329 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-model-cache\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:38.940616 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:38.940336 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddc034c0-93d8-4bfc-bd0b-878660843808-home\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:39:39.128258 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:39.128233 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n"] Jul 02 23:39:39.131724 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:39.131702 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/auth-test-model-83f2de-kserve-668fd74db7-lgn4n"] Jul 02 23:39:39.469105 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:39.469020 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" path="/var/lib/kubelet/pods/a2b9a6fe-877a-4b14-8470-2b0ce21a54e4/volumes" Jul 02 23:39:39.469482 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:39.469466 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddc034c0-93d8-4bfc-bd0b-878660843808" path="/var/lib/kubelet/pods/ddc034c0-93d8-4bfc-bd0b-878660843808/volumes" Jul 02 23:39:41.290238 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:41.290203 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:39:47.818298 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:47.818260 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:39:56.104421 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.104386 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng"] Jul 02 23:39:56.104813 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.104762 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" containerName="main" Jul 02 23:39:56.104813 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.104775 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" containerName="main" Jul 02 23:39:56.104813 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.104789 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddc034c0-93d8-4bfc-bd0b-878660843808" containerName="main" Jul 02 23:39:56.104813 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.104794 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddc034c0-93d8-4bfc-bd0b-878660843808" containerName="main" Jul 02 23:39:56.104945 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.104859 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddc034c0-93d8-4bfc-bd0b-878660843808" containerName="main" Jul 02 23:39:56.104945 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.104868 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2b9a6fe-877a-4b14-8470-2b0ce21a54e4" containerName="main" Jul 02 23:39:56.107776 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.107757 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.111344 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.111322 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"kube-root-ca.crt\"" Jul 02 23:39:56.111468 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.111324 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"openshift-service-ca.crt\"" Jul 02 23:39:56.111468 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.111324 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"default-dockercfg-8qfd8\"" Jul 02 23:39:56.111468 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.111379 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"sub-test-model-83f2de-kserve-self-signed-certs\"" Jul 02 23:39:56.116066 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.116046 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng"] Jul 02 23:39:56.178555 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.178521 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-home\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.178739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.178627 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0dcf434d-86f7-4fa5-ac07-a562839af77d-tls-certs\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.178739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.178655 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2962l\" (UniqueName: \"kubernetes.io/projected/0dcf434d-86f7-4fa5-ac07-a562839af77d-kube-api-access-2962l\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.178739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.178689 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-model-cache\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.178739 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.178733 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-dshm\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.178913 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.178757 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-tmp-dir\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.180628 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.180603 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:39:56.280195 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.280166 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-home\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.280336 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.280216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0dcf434d-86f7-4fa5-ac07-a562839af77d-tls-certs\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.280336 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.280234 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2962l\" (UniqueName: \"kubernetes.io/projected/0dcf434d-86f7-4fa5-ac07-a562839af77d-kube-api-access-2962l\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.280336 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.280263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-model-cache\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.280336 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.280290 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-dshm\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.280336 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.280307 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-tmp-dir\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.280702 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.280667 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-model-cache\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.280702 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.280694 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-home\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.280869 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.280717 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-tmp-dir\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.282585 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.282538 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-dshm\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.282860 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.282841 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0dcf434d-86f7-4fa5-ac07-a562839af77d-tls-certs\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.289056 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.289038 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2962l\" (UniqueName: \"kubernetes.io/projected/0dcf434d-86f7-4fa5-ac07-a562839af77d-kube-api-access-2962l\") pod \"sub-test-model-83f2de-kserve-6b4d5c984c-7mlng\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.417483 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.417455 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:39:56.545380 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.545357 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng"] Jul 02 23:39:56.547207 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:39:56.547173 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0dcf434d_86f7_4fa5_ac07_a562839af77d.slice/crio-171ce33a1c71698bc214c16f83dd6c7d6fb5e8011a3dada160263fc3b50ba424 WatchSource:0}: Error finding container 171ce33a1c71698bc214c16f83dd6c7d6fb5e8011a3dada160263fc3b50ba424: Status 404 returned error can't find the container with id 171ce33a1c71698bc214c16f83dd6c7d6fb5e8011a3dada160263fc3b50ba424 Jul 02 23:39:56.898498 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.898462 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" event={"ID":"0dcf434d-86f7-4fa5-ac07-a562839af77d","Type":"ContainerStarted","Data":"ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd"} Jul 02 23:39:56.898730 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.898504 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" event={"ID":"0dcf434d-86f7-4fa5-ac07-a562839af77d","Type":"ContainerStarted","Data":"171ce33a1c71698bc214c16f83dd6c7d6fb5e8011a3dada160263fc3b50ba424"} Jul 02 23:39:56.916147 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:56.916101 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" podStartSLOduration=0.9160852 podStartE2EDuration="916.0852ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:39:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:39:56.914619738 +0000 UTC m=+2303.987872103" watchObservedRunningTime="2026-07-02 23:39:56.9160852 +0000 UTC m=+2303.989337562" Jul 02 23:39:59.897052 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:39:59.897016 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:40:06.417707 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:06.417668 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:40:06.418095 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:06.417729 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:40:06.430484 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:06.430460 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:40:06.958825 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:06.958798 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:40:18.793391 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:18.793351 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:40:32.596270 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:32.596235 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:40:37.584336 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:37.584304 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv"] Jul 02 23:40:37.601784 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:37.601741 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:40:38.381060 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:38.381024 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng"] Jul 02 23:40:38.404780 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:38.404746 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:40:40.417882 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:40.417832 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:40:40.515052 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:40.515010 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:40:50.286545 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:50.286506 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:40:52.604471 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.604422 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" podUID="0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" containerName="main" containerID="cri-o://cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf" gracePeriod=45 Jul 02 23:40:52.859085 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.859028 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:40:52.949376 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949352 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tmp-dir\") pod \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " Jul 02 23:40:52.949504 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949386 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-model-cache\") pod \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " Jul 02 23:40:52.949504 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949440 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-dshm\") pod \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " Jul 02 23:40:52.949504 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949475 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vj7g\" (UniqueName: \"kubernetes.io/projected/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-kube-api-access-4vj7g\") pod \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " Jul 02 23:40:52.949504 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949497 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tls-certs\") pod \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " Jul 02 23:40:52.949753 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949590 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-home\") pod \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\" (UID: \"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9\") " Jul 02 23:40:52.949753 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949650 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" (UID: "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:52.949871 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949751 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" (UID: "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:52.949871 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949841 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-home" (OuterVolumeSpecName: "home") pod "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" (UID: "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:52.949961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949920 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-home\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:52.949961 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949943 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tmp-dir\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:52.950024 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.949961 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-model-cache\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:52.951606 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.951581 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-kube-api-access-4vj7g" (OuterVolumeSpecName: "kube-api-access-4vj7g") pod "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" (UID: "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9"). InnerVolumeSpecName "kube-api-access-4vj7g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:40:52.951702 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.951681 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-dshm" (OuterVolumeSpecName: "dshm") pod "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" (UID: "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:52.951746 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:52.951684 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" (UID: "0743a3e7-6a3a-4cbb-8e9b-4400d90174d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:40:53.050652 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.050629 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vj7g\" (UniqueName: \"kubernetes.io/projected/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-kube-api-access-4vj7g\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:53.050652 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.050651 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-tls-certs\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:53.050778 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.050664 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9-dshm\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:53.180170 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.180146 2568 generic.go:356] "Generic (PLEG): container finished" podID="0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" containerID="cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf" exitCode=0 Jul 02 23:40:53.180280 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.180189 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" Jul 02 23:40:53.180280 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.180224 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" event={"ID":"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9","Type":"ContainerDied","Data":"cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf"} Jul 02 23:40:53.180280 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.180253 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv" event={"ID":"0743a3e7-6a3a-4cbb-8e9b-4400d90174d9","Type":"ContainerDied","Data":"7f6f38caa2862303ab1ff1a0848f88d7d40d1aa39e28686f1de03ddf02e25bea"} Jul 02 23:40:53.180280 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.180269 2568 scope.go:117] "RemoveContainer" containerID="cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf" Jul 02 23:40:53.188848 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.188832 2568 scope.go:117] "RemoveContainer" containerID="cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf" Jul 02 23:40:53.189102 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:40:53.189087 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf\": container with ID starting with cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf not found: ID does not exist" containerID="cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf" Jul 02 23:40:53.189168 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.189110 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf"} err="failed to get container status \"cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf\": rpc error: code = NotFound desc = could not find container \"cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf\": container with ID starting with cb76ff1788d28a03c95ac3e9a4eba7a92f4eb7d16d1ff7e0021fa0dd7a713eaf not found: ID does not exist" Jul 02 23:40:53.200560 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.200539 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv"] Jul 02 23:40:53.204291 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.204270 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/sub-test-model-9352c3-kserve-6fb7b5c8f-dhmxv"] Jul 02 23:40:53.293255 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.293230 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6"] Jul 02 23:40:53.293586 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.293561 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" containerName="main" Jul 02 23:40:53.293635 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.293588 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" containerName="main" Jul 02 23:40:53.293687 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.293678 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" containerName="main" Jul 02 23:40:53.297766 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.297751 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.300139 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.300118 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9352c3\"/\"default-dockercfg-zb6ps\"" Jul 02 23:40:53.300232 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.300126 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9352c3\"/\"kube-root-ca.crt\"" Jul 02 23:40:53.300232 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.300156 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9352c3\"/\"rate-test-model-9352c3-kserve-self-signed-certs\"" Jul 02 23:40:53.300232 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.300208 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9352c3\"/\"openshift-service-ca.crt\"" Jul 02 23:40:53.304616 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.304598 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6"] Jul 02 23:40:53.323724 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.323703 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:40:53.352924 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.352900 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-model-cache\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.353009 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.352927 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-dshm\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.353009 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.352960 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tls-certs\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.353009 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.352979 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlmjn\" (UniqueName: \"kubernetes.io/projected/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-kube-api-access-tlmjn\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.353009 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.353007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-home\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.353154 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.353042 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tmp-dir\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.402589 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.402543 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" podUID="0dcf434d-86f7-4fa5-ac07-a562839af77d" containerName="main" containerID="cri-o://ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd" gracePeriod=45 Jul 02 23:40:53.453659 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.453601 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-model-cache\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.453659 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.453638 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-dshm\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.453857 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.453691 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tls-certs\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.453857 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.453722 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlmjn\" (UniqueName: \"kubernetes.io/projected/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-kube-api-access-tlmjn\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.453857 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.453848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-home\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.454012 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.453913 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tmp-dir\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.454012 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.453944 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-model-cache\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.454180 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.454157 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-home\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.454180 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.454166 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tmp-dir\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.455688 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.455670 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-dshm\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.456024 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.456007 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tls-certs\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.461381 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.461364 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlmjn\" (UniqueName: \"kubernetes.io/projected/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-kube-api-access-tlmjn\") pod \"rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.467740 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.467717 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0743a3e7-6a3a-4cbb-8e9b-4400d90174d9" path="/var/lib/kubelet/pods/0743a3e7-6a3a-4cbb-8e9b-4400d90174d9/volumes" Jul 02 23:40:53.609751 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.609718 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:40:53.650995 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.650972 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:40:53.742765 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.742741 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6"] Jul 02 23:40:53.744307 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:40:53.744279 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fc8ead0_a36b_46e2_8778_1e52b1ccc0ec.slice/crio-5e1b332e29c264a79d4dce3691a400b18def49f9e4b108a839cd76ba4aa0739c WatchSource:0}: Error finding container 5e1b332e29c264a79d4dce3691a400b18def49f9e4b108a839cd76ba4aa0739c: Status 404 returned error can't find the container with id 5e1b332e29c264a79d4dce3691a400b18def49f9e4b108a839cd76ba4aa0739c Jul 02 23:40:53.755506 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.755484 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2962l\" (UniqueName: \"kubernetes.io/projected/0dcf434d-86f7-4fa5-ac07-a562839af77d-kube-api-access-2962l\") pod \"0dcf434d-86f7-4fa5-ac07-a562839af77d\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " Jul 02 23:40:53.755619 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.755530 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-tmp-dir\") pod \"0dcf434d-86f7-4fa5-ac07-a562839af77d\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " Jul 02 23:40:53.755619 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.755579 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0dcf434d-86f7-4fa5-ac07-a562839af77d-tls-certs\") pod \"0dcf434d-86f7-4fa5-ac07-a562839af77d\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " Jul 02 23:40:53.755701 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.755655 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-home\") pod \"0dcf434d-86f7-4fa5-ac07-a562839af77d\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " Jul 02 23:40:53.755733 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.755715 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-model-cache\") pod \"0dcf434d-86f7-4fa5-ac07-a562839af77d\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " Jul 02 23:40:53.755776 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.755753 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-dshm\") pod \"0dcf434d-86f7-4fa5-ac07-a562839af77d\" (UID: \"0dcf434d-86f7-4fa5-ac07-a562839af77d\") " Jul 02 23:40:53.755831 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.755802 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0dcf434d-86f7-4fa5-ac07-a562839af77d" (UID: "0dcf434d-86f7-4fa5-ac07-a562839af77d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:53.755975 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.755949 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-home" (OuterVolumeSpecName: "home") pod "0dcf434d-86f7-4fa5-ac07-a562839af77d" (UID: "0dcf434d-86f7-4fa5-ac07-a562839af77d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:53.755975 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.755958 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-model-cache" (OuterVolumeSpecName: "model-cache") pod "0dcf434d-86f7-4fa5-ac07-a562839af77d" (UID: "0dcf434d-86f7-4fa5-ac07-a562839af77d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:53.756142 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.756089 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-home\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:53.756142 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.756106 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-model-cache\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:53.756142 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.756120 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-tmp-dir\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:53.757759 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.757732 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dcf434d-86f7-4fa5-ac07-a562839af77d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0dcf434d-86f7-4fa5-ac07-a562839af77d" (UID: "0dcf434d-86f7-4fa5-ac07-a562839af77d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:40:53.758030 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.758008 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dcf434d-86f7-4fa5-ac07-a562839af77d-kube-api-access-2962l" (OuterVolumeSpecName: "kube-api-access-2962l") pod "0dcf434d-86f7-4fa5-ac07-a562839af77d" (UID: "0dcf434d-86f7-4fa5-ac07-a562839af77d"). InnerVolumeSpecName "kube-api-access-2962l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:40:53.758030 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.758024 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-dshm" (OuterVolumeSpecName: "dshm") pod "0dcf434d-86f7-4fa5-ac07-a562839af77d" (UID: "0dcf434d-86f7-4fa5-ac07-a562839af77d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:40:53.857285 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.857258 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0dcf434d-86f7-4fa5-ac07-a562839af77d-dshm\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:53.857285 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.857284 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2962l\" (UniqueName: \"kubernetes.io/projected/0dcf434d-86f7-4fa5-ac07-a562839af77d-kube-api-access-2962l\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:53.857445 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:53.857295 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0dcf434d-86f7-4fa5-ac07-a562839af77d-tls-certs\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:40:54.187647 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.187604 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" event={"ID":"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec","Type":"ContainerStarted","Data":"64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760"} Jul 02 23:40:54.187647 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.187655 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" event={"ID":"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec","Type":"ContainerStarted","Data":"5e1b332e29c264a79d4dce3691a400b18def49f9e4b108a839cd76ba4aa0739c"} Jul 02 23:40:54.188902 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.188878 2568 generic.go:356] "Generic (PLEG): container finished" podID="0dcf434d-86f7-4fa5-ac07-a562839af77d" containerID="ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd" exitCode=0 Jul 02 23:40:54.189033 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.188918 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" event={"ID":"0dcf434d-86f7-4fa5-ac07-a562839af77d","Type":"ContainerDied","Data":"ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd"} Jul 02 23:40:54.189033 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.188924 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" Jul 02 23:40:54.189033 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.188943 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng" event={"ID":"0dcf434d-86f7-4fa5-ac07-a562839af77d","Type":"ContainerDied","Data":"171ce33a1c71698bc214c16f83dd6c7d6fb5e8011a3dada160263fc3b50ba424"} Jul 02 23:40:54.189033 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.188965 2568 scope.go:117] "RemoveContainer" containerID="ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd" Jul 02 23:40:54.198460 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.198441 2568 scope.go:117] "RemoveContainer" containerID="ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd" Jul 02 23:40:54.198718 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:40:54.198698 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd\": container with ID starting with ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd not found: ID does not exist" containerID="ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd" Jul 02 23:40:54.198772 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.198726 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd"} err="failed to get container status \"ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd\": rpc error: code = NotFound desc = could not find container \"ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd\": container with ID starting with ae0ced3c446f787dd83a8f521aadcd6146ac66ca857d019f185c1d7bb5eeffdd not found: ID does not exist" Jul 02 23:40:54.206890 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.206849 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" podStartSLOduration=1.206835571 podStartE2EDuration="1.206835571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:40:54.204305742 +0000 UTC m=+2361.277558105" watchObservedRunningTime="2026-07-02 23:40:54.206835571 +0000 UTC m=+2361.280087934" Jul 02 23:40:54.219321 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.219293 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng"] Jul 02 23:40:54.223176 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:54.223153 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/sub-test-model-83f2de-kserve-6b4d5c984c-7mlng"] Jul 02 23:40:55.471375 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:40:55.471341 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dcf434d-86f7-4fa5-ac07-a562839af77d" path="/var/lib/kubelet/pods/0dcf434d-86f7-4fa5-ac07-a562839af77d/volumes" Jul 02 23:41:03.610538 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:03.610496 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:41:03.611084 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:03.610549 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:41:03.622723 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:03.622695 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:41:04.253343 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:04.253314 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:41:04.990909 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:04.990873 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:41:09.477766 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:09.477733 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:41:09.533015 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:09.532983 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:41:19.693906 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.693865 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz"] Jul 02 23:41:19.694358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.694217 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dcf434d-86f7-4fa5-ac07-a562839af77d" containerName="main" Jul 02 23:41:19.694358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.694229 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dcf434d-86f7-4fa5-ac07-a562839af77d" containerName="main" Jul 02 23:41:19.694358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.694301 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dcf434d-86f7-4fa5-ac07-a562839af77d" containerName="main" Jul 02 23:41:19.697180 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.697160 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.700936 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.700914 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"rate-test-model-83f2de-kserve-self-signed-certs\"" Jul 02 23:41:19.701066 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.700939 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"default-dockercfg-8qfd8\"" Jul 02 23:41:19.701066 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.700920 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"kube-root-ca.crt\"" Jul 02 23:41:19.701066 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.700923 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-83f2de\"/\"openshift-service-ca.crt\"" Jul 02 23:41:19.706226 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.706205 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz"] Jul 02 23:41:19.721686 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.721660 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:41:19.797150 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.797102 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-dshm\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.797331 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.797160 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-tmp-dir\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.797331 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.797205 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b74342a6-0d7f-420f-b5a4-c0d20129e127-tls-certs\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.797331 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.797234 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-home\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.797331 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.797317 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n95sv\" (UniqueName: \"kubernetes.io/projected/b74342a6-0d7f-420f-b5a4-c0d20129e127-kube-api-access-n95sv\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.797524 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.797362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-model-cache\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.898819 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.898784 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-model-cache\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.899019 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.898849 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-dshm\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.899019 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.898882 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-tmp-dir\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.899019 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.898913 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b74342a6-0d7f-420f-b5a4-c0d20129e127-tls-certs\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.899019 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.898940 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-home\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.899019 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.898964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95sv\" (UniqueName: \"kubernetes.io/projected/b74342a6-0d7f-420f-b5a4-c0d20129e127-kube-api-access-n95sv\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.899288 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.899238 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-model-cache\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.899288 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.899273 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-tmp-dir\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.899406 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.899372 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-home\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.901059 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.901041 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-dshm\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.901419 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.901402 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b74342a6-0d7f-420f-b5a4-c0d20129e127-tls-certs\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:19.907107 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:19.907086 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n95sv\" (UniqueName: \"kubernetes.io/projected/b74342a6-0d7f-420f-b5a4-c0d20129e127-kube-api-access-n95sv\") pod \"rate-test-model-83f2de-kserve-5d844d6b67-5knhz\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:20.006992 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:20.006891 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:20.140227 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:20.140197 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz"] Jul 02 23:41:20.141327 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:41:20.141299 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb74342a6_0d7f_420f_b5a4_c0d20129e127.slice/crio-760926afe00728fd970bad0b5aca8c934b14f0269313534f8b31b2ee129778ef WatchSource:0}: Error finding container 760926afe00728fd970bad0b5aca8c934b14f0269313534f8b31b2ee129778ef: Status 404 returned error can't find the container with id 760926afe00728fd970bad0b5aca8c934b14f0269313534f8b31b2ee129778ef Jul 02 23:41:20.330880 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:20.330780 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" event={"ID":"b74342a6-0d7f-420f-b5a4-c0d20129e127","Type":"ContainerStarted","Data":"d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6"} Jul 02 23:41:20.330880 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:20.330828 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" event={"ID":"b74342a6-0d7f-420f-b5a4-c0d20129e127","Type":"ContainerStarted","Data":"760926afe00728fd970bad0b5aca8c934b14f0269313534f8b31b2ee129778ef"} Jul 02 23:41:20.351616 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:20.351542 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" podStartSLOduration=1.35152621 podStartE2EDuration="1.35152621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:41:20.350441276 +0000 UTC m=+2387.423693637" watchObservedRunningTime="2026-07-02 23:41:20.35152621 +0000 UTC m=+2387.424778551" Jul 02 23:41:24.104819 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:24.104784 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:41:30.007362 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:30.007328 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:30.007877 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:30.007376 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:30.019719 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:30.019691 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:30.394551 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:30.394519 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:41:32.020546 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:32.020508 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:41:32.121101 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:32.121065 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:41:33.596305 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:33.596274 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:41:33.603266 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:33.603243 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmwh5_17b1942c-8c3b-4510-bee8-32742e97493a/ovn-acl-logging/0.log" Jul 02 23:41:40.980209 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:40.980176 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:41:41.081825 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:41.081764 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:41:50.787495 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:41:50.787426 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:42:04.093146 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:04.093051 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:42:10.572721 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:10.572688 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6"] Jul 02 23:42:10.595232 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:10.595204 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:11.371690 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:11.371659 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz"] Jul 02 23:42:11.392530 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:11.392498 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:12.530599 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:12.530549 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:12.884304 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:12.884269 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:12.984054 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:12.984018 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:13.083856 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:13.083820 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:13.184422 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:13.184340 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:16.982071 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:16.982034 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:42:17.954836 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:17.954795 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:25.592487 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.592426 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" podUID="3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" containerName="main" containerID="cri-o://64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760" gracePeriod=45 Jul 02 23:42:25.845374 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.845318 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:42:25.958773 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.958744 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-model-cache\") pod \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " Jul 02 23:42:25.958773 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.958774 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-dshm\") pod \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " Jul 02 23:42:25.958975 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.958796 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlmjn\" (UniqueName: \"kubernetes.io/projected/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-kube-api-access-tlmjn\") pod \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " Jul 02 23:42:25.958975 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.958815 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tls-certs\") pod \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " Jul 02 23:42:25.958975 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.958878 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-home\") pod \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " Jul 02 23:42:25.958975 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.958895 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tmp-dir\") pod \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\" (UID: \"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec\") " Jul 02 23:42:25.959196 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.959015 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-model-cache" (OuterVolumeSpecName: "model-cache") pod "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" (UID: "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:25.959196 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.959131 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-home" (OuterVolumeSpecName: "home") pod "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" (UID: "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:25.959276 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.959223 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" (UID: "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:25.959322 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.959288 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-model-cache\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:25.959322 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.959310 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-home\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:25.960920 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.960894 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-dshm" (OuterVolumeSpecName: "dshm") pod "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" (UID: "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:25.961035 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.960937 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" (UID: "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:42:25.961035 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:25.960988 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-kube-api-access-tlmjn" (OuterVolumeSpecName: "kube-api-access-tlmjn") pod "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" (UID: "3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec"). InnerVolumeSpecName "kube-api-access-tlmjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:42:26.060080 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.060051 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-dshm\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:26.060080 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.060077 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlmjn\" (UniqueName: \"kubernetes.io/projected/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-kube-api-access-tlmjn\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:26.060221 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.060093 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tls-certs\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:26.060221 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.060106 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec-tmp-dir\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:26.391741 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.391691 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" podUID="b74342a6-0d7f-420f-b5a4-c0d20129e127" containerName="main" containerID="cri-o://d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6" gracePeriod=45 Jul 02 23:42:26.639864 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.639842 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:42:26.687350 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.687265 2568 generic.go:356] "Generic (PLEG): container finished" podID="b74342a6-0d7f-420f-b5a4-c0d20129e127" containerID="d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6" exitCode=0 Jul 02 23:42:26.687350 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.687297 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" event={"ID":"b74342a6-0d7f-420f-b5a4-c0d20129e127","Type":"ContainerDied","Data":"d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6"} Jul 02 23:42:26.687350 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.687334 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" event={"ID":"b74342a6-0d7f-420f-b5a4-c0d20129e127","Type":"ContainerDied","Data":"760926afe00728fd970bad0b5aca8c934b14f0269313534f8b31b2ee129778ef"} Jul 02 23:42:26.687350 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.687352 2568 scope.go:117] "RemoveContainer" containerID="d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6" Jul 02 23:42:26.687704 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.687308 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz" Jul 02 23:42:26.688677 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.688655 2568 generic.go:356] "Generic (PLEG): container finished" podID="3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" containerID="64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760" exitCode=0 Jul 02 23:42:26.688851 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.688698 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" Jul 02 23:42:26.688953 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.688714 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" event={"ID":"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec","Type":"ContainerDied","Data":"64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760"} Jul 02 23:42:26.688953 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.688945 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6" event={"ID":"3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec","Type":"ContainerDied","Data":"5e1b332e29c264a79d4dce3691a400b18def49f9e4b108a839cd76ba4aa0739c"} Jul 02 23:42:26.698885 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.698858 2568 scope.go:117] "RemoveContainer" containerID="d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6" Jul 02 23:42:26.699170 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:42:26.699151 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6\": container with ID starting with d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6 not found: ID does not exist" containerID="d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6" Jul 02 23:42:26.699269 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.699177 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6"} err="failed to get container status \"d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6\": rpc error: code = NotFound desc = could not find container \"d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6\": container with ID starting with d3954113a6de716d545aee9298ad632f64245819e3b69f64a20103ea6fdd54e6 not found: ID does not exist" Jul 02 23:42:26.699269 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.699194 2568 scope.go:117] "RemoveContainer" containerID="64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760" Jul 02 23:42:26.706857 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.706838 2568 scope.go:117] "RemoveContainer" containerID="64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760" Jul 02 23:42:26.707072 ip-10-0-136-71 kubenswrapper[2568]: E0702 23:42:26.707054 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760\": container with ID starting with 64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760 not found: ID does not exist" containerID="64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760" Jul 02 23:42:26.707135 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.707081 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760"} err="failed to get container status \"64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760\": rpc error: code = NotFound desc = could not find container \"64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760\": container with ID starting with 64c5d845839a4dee2f2119805dc2ff05752776938f1a4569f225136ce3270760 not found: ID does not exist" Jul 02 23:42:26.712545 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.712522 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6"] Jul 02 23:42:26.716406 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.716385 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9352c3/rate-test-model-9352c3-kserve-5fcb88cbc5-jjwh6"] Jul 02 23:42:26.766487 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.766465 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b74342a6-0d7f-420f-b5a4-c0d20129e127-tls-certs\") pod \"b74342a6-0d7f-420f-b5a4-c0d20129e127\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " Jul 02 23:42:26.766601 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.766493 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n95sv\" (UniqueName: \"kubernetes.io/projected/b74342a6-0d7f-420f-b5a4-c0d20129e127-kube-api-access-n95sv\") pod \"b74342a6-0d7f-420f-b5a4-c0d20129e127\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " Jul 02 23:42:26.766601 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.766524 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-home\") pod \"b74342a6-0d7f-420f-b5a4-c0d20129e127\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " Jul 02 23:42:26.766601 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.766584 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-dshm\") pod \"b74342a6-0d7f-420f-b5a4-c0d20129e127\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " Jul 02 23:42:26.766783 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.766605 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-tmp-dir\") pod \"b74342a6-0d7f-420f-b5a4-c0d20129e127\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " Jul 02 23:42:26.766783 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.766659 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-model-cache\") pod \"b74342a6-0d7f-420f-b5a4-c0d20129e127\" (UID: \"b74342a6-0d7f-420f-b5a4-c0d20129e127\") " Jul 02 23:42:26.766882 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.766800 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-home" (OuterVolumeSpecName: "home") pod "b74342a6-0d7f-420f-b5a4-c0d20129e127" (UID: "b74342a6-0d7f-420f-b5a4-c0d20129e127"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:26.766940 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.766884 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b74342a6-0d7f-420f-b5a4-c0d20129e127" (UID: "b74342a6-0d7f-420f-b5a4-c0d20129e127"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:26.766987 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.766963 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-home\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:26.766987 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.766967 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-model-cache" (OuterVolumeSpecName: "model-cache") pod "b74342a6-0d7f-420f-b5a4-c0d20129e127" (UID: "b74342a6-0d7f-420f-b5a4-c0d20129e127"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:26.767095 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.766979 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-tmp-dir\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:26.768460 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.768440 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-dshm" (OuterVolumeSpecName: "dshm") pod "b74342a6-0d7f-420f-b5a4-c0d20129e127" (UID: "b74342a6-0d7f-420f-b5a4-c0d20129e127"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:26.768546 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.768504 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b74342a6-0d7f-420f-b5a4-c0d20129e127-kube-api-access-n95sv" (OuterVolumeSpecName: "kube-api-access-n95sv") pod "b74342a6-0d7f-420f-b5a4-c0d20129e127" (UID: "b74342a6-0d7f-420f-b5a4-c0d20129e127"). InnerVolumeSpecName "kube-api-access-n95sv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:42:26.768624 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.768613 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b74342a6-0d7f-420f-b5a4-c0d20129e127-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b74342a6-0d7f-420f-b5a4-c0d20129e127" (UID: "b74342a6-0d7f-420f-b5a4-c0d20129e127"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:42:26.868124 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.868096 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-dshm\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:26.868124 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.868119 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b74342a6-0d7f-420f-b5a4-c0d20129e127-model-cache\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:26.868262 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.868134 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b74342a6-0d7f-420f-b5a4-c0d20129e127-tls-certs\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:26.868262 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:26.868146 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n95sv\" (UniqueName: \"kubernetes.io/projected/b74342a6-0d7f-420f-b5a4-c0d20129e127-kube-api-access-n95sv\") on node \"ip-10-0-136-71.ec2.internal\" DevicePath \"\"" Jul 02 23:42:27.008991 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:27.008966 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz"] Jul 02 23:42:27.014156 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:27.014135 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-83f2de/rate-test-model-83f2de-kserve-5d844d6b67-5knhz"] Jul 02 23:42:27.468346 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:27.468313 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" path="/var/lib/kubelet/pods/3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec/volumes" Jul 02 23:42:27.468787 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:27.468766 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b74342a6-0d7f-420f-b5a4-c0d20129e127" path="/var/lib/kubelet/pods/b74342a6-0d7f-420f-b5a4-c0d20129e127/volumes" Jul 02 23:42:38.261883 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:38.261837 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:39.324015 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:39.323970 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:39.425226 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:39.425196 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:39.526814 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:39.526785 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:42.388941 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:42.388908 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b64kb"] Jul 02 23:42:45.263476 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:45.263437 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:45.363815 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:45.363780 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:45.510203 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:45.510170 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:45.729724 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:45.729692 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:42:46.141134 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:46.141094 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7444d59b49-q9z55_dabbd3d9-a17b-4ab8-b536-5f15d4955247/authorino/0.log" Jul 02 23:42:51.435300 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:51.435267 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-4xvtf_68e8c8db-a45b-47f0-8918-1ece0183f62e/manager/0.log" Jul 02 23:42:51.794323 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:51.794255 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-676b9dcdc-q5zgq_3e845ee6-2bd5-4ef2-bbc0-445166acc977/copy-manifests/0.log" Jul 02 23:42:51.800106 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:51.800082 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-676b9dcdc-q5zgq_3e845ee6-2bd5-4ef2-bbc0-445166acc977/manager/0.log" Jul 02 23:42:52.651786 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:52.651758 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5_6aed00e5-2462-4f86-adb2-4acde41936a5/util/0.log" Jul 02 23:42:52.657490 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:52.657467 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5_6aed00e5-2462-4f86-adb2-4acde41936a5/pull/0.log" Jul 02 23:42:52.664932 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:52.664915 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5_6aed00e5-2462-4f86-adb2-4acde41936a5/extract/0.log" Jul 02 23:42:52.770232 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:52.770214 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp_7653d4d8-35e7-4339-b475-109fe9b64f2b/util/0.log" Jul 02 23:42:52.776235 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:52.776216 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp_7653d4d8-35e7-4339-b475-109fe9b64f2b/pull/0.log" Jul 02 23:42:52.782031 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:52.782016 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp_7653d4d8-35e7-4339-b475-109fe9b64f2b/extract/0.log" Jul 02 23:42:52.884004 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:52.883964 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw_0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2/util/0.log" Jul 02 23:42:52.889860 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:52.889844 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw_0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2/pull/0.log" Jul 02 23:42:52.895402 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:52.895387 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw_0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2/extract/0.log" Jul 02 23:42:53.000547 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:53.000495 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz_73ef1068-e9f1-47cf-8449-ceac273f427e/extract/0.log" Jul 02 23:42:53.005851 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:53.005835 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz_73ef1068-e9f1-47cf-8449-ceac273f427e/util/0.log" Jul 02 23:42:53.011251 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:53.011237 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz_73ef1068-e9f1-47cf-8449-ceac273f427e/pull/0.log" Jul 02 23:42:53.129201 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:53.129183 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7444d59b49-q9z55_dabbd3d9-a17b-4ab8-b536-5f15d4955247/authorino/0.log" Jul 02 23:42:53.826541 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:53.826510 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b64kb_f7024c9e-cd75-47fc-8b56-526b422b7194/limitador/0.log" Jul 02 23:42:54.620788 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:54.620760 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-99946bbb8-tg2t5_d9a1431f-25ad-48b5-b658-86f757f2f5a1/kube-auth-proxy/0.log" Jul 02 23:42:55.945413 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:42:55.945380 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-6z7w2_d9c0699c-1740-4355-982e-e5fde171b0a4/main/0.log" Jul 02 23:43:00.308926 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.308895 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dblpz/must-gather-jtcpt"] Jul 02 23:43:00.309280 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.309224 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b74342a6-0d7f-420f-b5a4-c0d20129e127" containerName="main" Jul 02 23:43:00.309280 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.309236 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b74342a6-0d7f-420f-b5a4-c0d20129e127" containerName="main" Jul 02 23:43:00.309280 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.309256 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" containerName="main" Jul 02 23:43:00.309280 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.309263 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" containerName="main" Jul 02 23:43:00.309445 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.309310 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b74342a6-0d7f-420f-b5a4-c0d20129e127" containerName="main" Jul 02 23:43:00.309445 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.309318 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc8ead0-a36b-46e2-8778-1e52b1ccc0ec" containerName="main" Jul 02 23:43:00.312612 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.312591 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dblpz/must-gather-jtcpt" Jul 02 23:43:00.315160 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.315138 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dblpz\"/\"default-dockercfg-27qc6\"" Jul 02 23:43:00.315288 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.315263 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dblpz\"/\"kube-root-ca.crt\"" Jul 02 23:43:00.315350 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.315312 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dblpz\"/\"openshift-service-ca.crt\"" Jul 02 23:43:00.321266 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.321242 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dblpz/must-gather-jtcpt"] Jul 02 23:43:00.328526 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.328501 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:43:00.416193 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.416172 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6krs5\" (UniqueName: \"kubernetes.io/projected/13e0bb9d-df02-4845-850d-bd4f75c91a51-kube-api-access-6krs5\") pod \"must-gather-jtcpt\" (UID: \"13e0bb9d-df02-4845-850d-bd4f75c91a51\") " pod="openshift-must-gather-dblpz/must-gather-jtcpt" Jul 02 23:43:00.416338 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.416236 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/13e0bb9d-df02-4845-850d-bd4f75c91a51-must-gather-output\") pod \"must-gather-jtcpt\" (UID: \"13e0bb9d-df02-4845-850d-bd4f75c91a51\") " pod="openshift-must-gather-dblpz/must-gather-jtcpt" Jul 02 23:43:00.428870 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.428846 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" limit=1024 Jul 02 23:43:00.517165 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.517141 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6krs5\" (UniqueName: \"kubernetes.io/projected/13e0bb9d-df02-4845-850d-bd4f75c91a51-kube-api-access-6krs5\") pod \"must-gather-jtcpt\" (UID: \"13e0bb9d-df02-4845-850d-bd4f75c91a51\") " pod="openshift-must-gather-dblpz/must-gather-jtcpt" Jul 02 23:43:00.517270 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.517199 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/13e0bb9d-df02-4845-850d-bd4f75c91a51-must-gather-output\") pod \"must-gather-jtcpt\" (UID: \"13e0bb9d-df02-4845-850d-bd4f75c91a51\") " pod="openshift-must-gather-dblpz/must-gather-jtcpt" Jul 02 23:43:00.517467 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.517451 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/13e0bb9d-df02-4845-850d-bd4f75c91a51-must-gather-output\") pod \"must-gather-jtcpt\" (UID: \"13e0bb9d-df02-4845-850d-bd4f75c91a51\") " pod="openshift-must-gather-dblpz/must-gather-jtcpt" Jul 02 23:43:00.526179 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.526157 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6krs5\" (UniqueName: \"kubernetes.io/projected/13e0bb9d-df02-4845-850d-bd4f75c91a51-kube-api-access-6krs5\") pod \"must-gather-jtcpt\" (UID: \"13e0bb9d-df02-4845-850d-bd4f75c91a51\") " pod="openshift-must-gather-dblpz/must-gather-jtcpt" Jul 02 23:43:00.622592 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.622552 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dblpz/must-gather-jtcpt" Jul 02 23:43:00.751437 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.751409 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dblpz/must-gather-jtcpt"] Jul 02 23:43:00.752978 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:43:00.752950 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13e0bb9d_df02_4845_850d_bd4f75c91a51.slice/crio-457c7c5c892e8c361f0fccbf50a52610415d5c55483281ddb28a5ead9e555dd6 WatchSource:0}: Error finding container 457c7c5c892e8c361f0fccbf50a52610415d5c55483281ddb28a5ead9e555dd6: Status 404 returned error can't find the container with id 457c7c5c892e8c361f0fccbf50a52610415d5c55483281ddb28a5ead9e555dd6 Jul 02 23:43:00.754865 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.754850 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:43:00.866091 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:00.866063 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dblpz/must-gather-jtcpt" event={"ID":"13e0bb9d-df02-4845-850d-bd4f75c91a51","Type":"ContainerStarted","Data":"457c7c5c892e8c361f0fccbf50a52610415d5c55483281ddb28a5ead9e555dd6"} Jul 02 23:43:01.876520 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:01.875710 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dblpz/must-gather-jtcpt" event={"ID":"13e0bb9d-df02-4845-850d-bd4f75c91a51","Type":"ContainerStarted","Data":"f8e8bc2a7df74b60415067e318da6f7eec4d664b7c1f75705665e600af3f857a"} Jul 02 23:43:01.876520 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:01.875766 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dblpz/must-gather-jtcpt" event={"ID":"13e0bb9d-df02-4845-850d-bd4f75c91a51","Type":"ContainerStarted","Data":"98c2910ccf34bbe2eecd6e5e0da30e9b82d68d1810eb15b1f846fa30bb2b0a1a"} Jul 02 23:43:03.139686 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:03.139637 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lnsbx_bb3efb95-e804-4cae-b3b5-220db506cfe8/global-pull-secret-syncer/0.log" Jul 02 23:43:03.192658 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:03.192622 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-77nbd_1dac54d6-06f0-4f56-b1c8-6e24e59ea5af/konnectivity-agent/0.log" Jul 02 23:43:03.287301 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:03.287255 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-71.ec2.internal_934cc245ae5a1b254602b20dfff22029/haproxy/0.log" Jul 02 23:43:07.664069 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.664037 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5_6aed00e5-2462-4f86-adb2-4acde41936a5/extract/0.log" Jul 02 23:43:07.684544 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.684519 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5_6aed00e5-2462-4f86-adb2-4acde41936a5/util/0.log" Jul 02 23:43:07.704995 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.704971 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vg8z5_6aed00e5-2462-4f86-adb2-4acde41936a5/pull/0.log" Jul 02 23:43:07.729461 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.729376 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp_7653d4d8-35e7-4339-b475-109fe9b64f2b/extract/0.log" Jul 02 23:43:07.753193 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.753118 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp_7653d4d8-35e7-4339-b475-109fe9b64f2b/util/0.log" Jul 02 23:43:07.776521 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.776495 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9gvp_7653d4d8-35e7-4339-b475-109fe9b64f2b/pull/0.log" Jul 02 23:43:07.816558 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.816499 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw_0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2/extract/0.log" Jul 02 23:43:07.837399 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.837375 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw_0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2/util/0.log" Jul 02 23:43:07.857539 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.857514 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xw8lw_0a19ad93-b86a-47e7-8e1d-c8172c6dd9a2/pull/0.log" Jul 02 23:43:07.885248 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.885203 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz_73ef1068-e9f1-47cf-8449-ceac273f427e/extract/0.log" Jul 02 23:43:07.908133 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.908057 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz_73ef1068-e9f1-47cf-8449-ceac273f427e/util/0.log" Jul 02 23:43:07.936124 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:07.936042 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jlglz_73ef1068-e9f1-47cf-8449-ceac273f427e/pull/0.log" Jul 02 23:43:08.363392 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:08.363328 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7444d59b49-q9z55_dabbd3d9-a17b-4ab8-b536-5f15d4955247/authorino/0.log" Jul 02 23:43:08.614428 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:08.614343 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b64kb_f7024c9e-cd75-47fc-8b56-526b422b7194/limitador/0.log" Jul 02 23:43:10.396805 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.395797 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67ad99e8-4256-47fc-a2fe-72ae15865c51/alertmanager/0.log" Jul 02 23:43:10.419376 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.419346 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67ad99e8-4256-47fc-a2fe-72ae15865c51/config-reloader/0.log" Jul 02 23:43:10.443082 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.442798 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67ad99e8-4256-47fc-a2fe-72ae15865c51/kube-rbac-proxy-web/0.log" Jul 02 23:43:10.473334 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.473301 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67ad99e8-4256-47fc-a2fe-72ae15865c51/kube-rbac-proxy/0.log" Jul 02 23:43:10.494491 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.494463 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67ad99e8-4256-47fc-a2fe-72ae15865c51/kube-rbac-proxy-metric/0.log" Jul 02 23:43:10.517723 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.517695 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67ad99e8-4256-47fc-a2fe-72ae15865c51/prom-label-proxy/0.log" Jul 02 23:43:10.538017 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.537991 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67ad99e8-4256-47fc-a2fe-72ae15865c51/init-config-reloader/0.log" Jul 02 23:43:10.600980 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.600890 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-4lmt2_095e97d7-a7b1-437e-a1e2-dadc8524eaf9/kube-state-metrics/0.log" Jul 02 23:43:10.621059 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.621021 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-4lmt2_095e97d7-a7b1-437e-a1e2-dadc8524eaf9/kube-rbac-proxy-main/0.log" Jul 02 23:43:10.642364 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.642323 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-4lmt2_095e97d7-a7b1-437e-a1e2-dadc8524eaf9/kube-rbac-proxy-self/0.log" Jul 02 23:43:10.669973 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.669931 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-67dcb868c9-w2hpl_0d829ca4-3ab1-4c14-a42f-d21a4dd13096/metrics-server/0.log" Jul 02 23:43:10.724948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.724905 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ghk6j_23521425-31dd-4d23-90ce-e855a6b5549b/node-exporter/0.log" Jul 02 23:43:10.747279 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.747254 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ghk6j_23521425-31dd-4d23-90ce-e855a6b5549b/kube-rbac-proxy/0.log" Jul 02 23:43:10.773675 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.773636 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ghk6j_23521425-31dd-4d23-90ce-e855a6b5549b/init-textfile/0.log" Jul 02 23:43:10.931769 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.931718 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hpdzb_7583ca97-2e44-462c-b17c-cc79dc6f8589/kube-rbac-proxy-main/0.log" Jul 02 23:43:10.951370 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.951345 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hpdzb_7583ca97-2e44-462c-b17c-cc79dc6f8589/kube-rbac-proxy-self/0.log" Jul 02 23:43:10.971355 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:10.971299 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hpdzb_7583ca97-2e44-462c-b17c-cc79dc6f8589/openshift-state-metrics/0.log" Jul 02 23:43:11.235012 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:11.234939 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-559fd44f86-v8nw5_a6f84056-2ca0-4eb4-9125-85cfde48a8ec/telemeter-client/0.log" Jul 02 23:43:11.255502 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:11.255472 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-559fd44f86-v8nw5_a6f84056-2ca0-4eb4-9125-85cfde48a8ec/reload/0.log" Jul 02 23:43:11.275599 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:11.275557 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-559fd44f86-v8nw5_a6f84056-2ca0-4eb4-9125-85cfde48a8ec/kube-rbac-proxy/0.log" Jul 02 23:43:13.141265 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.141187 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dblpz/must-gather-jtcpt" podStartSLOduration=12.350647057 podStartE2EDuration="13.141166396s" totalImagesPullingTime="790.519339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:43:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:43:01.893560874 +0000 UTC m=+2488.966813237" watchObservedRunningTime="2026-07-02 23:43:13.141166396 +0000 UTC m=+2500.214418760" Jul 02 23:43:13.144157 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.144129 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr"] Jul 02 23:43:13.149839 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.149813 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.155218 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.155190 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr"] Jul 02 23:43:13.244190 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.244145 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jz6s\" (UniqueName: \"kubernetes.io/projected/26194a4d-c5ab-49d4-a579-aa69c212ab03-kube-api-access-8jz6s\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.244558 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.244537 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-podres\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.244732 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.244715 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-proc\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.244890 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.244874 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-sys\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.245014 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.244999 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-lib-modules\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.346186 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.346149 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-sys\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.346434 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.346415 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-lib-modules\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.346590 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.346558 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jz6s\" (UniqueName: \"kubernetes.io/projected/26194a4d-c5ab-49d4-a579-aa69c212ab03-kube-api-access-8jz6s\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.346799 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.346781 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-podres\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.346948 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.346931 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-proc\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.347121 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.347093 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-sys\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.347285 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.347097 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-proc\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.347373 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.347033 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-podres\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.347472 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.347182 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26194a4d-c5ab-49d4-a579-aa69c212ab03-lib-modules\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.354945 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.354918 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jz6s\" (UniqueName: \"kubernetes.io/projected/26194a4d-c5ab-49d4-a579-aa69c212ab03-kube-api-access-8jz6s\") pod \"perf-node-gather-daemonset-5d5wr\" (UID: \"26194a4d-c5ab-49d4-a579-aa69c212ab03\") " pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.467064 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.466974 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.658625 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.658596 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr"] Jul 02 23:43:13.661776 ip-10-0-136-71 kubenswrapper[2568]: W0702 23:43:13.661747 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod26194a4d_c5ab_49d4_a579_aa69c212ab03.slice/crio-a3cf168139e2bc3df5081addc2e0abc480e1e077b8c26e2e1bdc363171091bc5 WatchSource:0}: Error finding container a3cf168139e2bc3df5081addc2e0abc480e1e077b8c26e2e1bdc363171091bc5: Status 404 returned error can't find the container with id a3cf168139e2bc3df5081addc2e0abc480e1e077b8c26e2e1bdc363171091bc5 Jul 02 23:43:13.758874 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.758850 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-rjnfd_cfbde320-130d-4761-81df-3154c7c0f7e2/download-server/0.log" Jul 02 23:43:13.964710 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.964672 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" event={"ID":"26194a4d-c5ab-49d4-a579-aa69c212ab03","Type":"ContainerStarted","Data":"0f7050644e70d5f73b5938daf5dcdd5ac631ab0bd5717c2056ec8798557a6889"} Jul 02 23:43:13.964883 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.964743 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" event={"ID":"26194a4d-c5ab-49d4-a579-aa69c212ab03","Type":"ContainerStarted","Data":"a3cf168139e2bc3df5081addc2e0abc480e1e077b8c26e2e1bdc363171091bc5"} Jul 02 23:43:13.964883 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.964778 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:13.983058 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:13.982950 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" podStartSLOduration=0.982928761 podStartE2EDuration="982.928761ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:43:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:43:13.980168817 +0000 UTC m=+2501.053421180" watchObservedRunningTime="2026-07-02 23:43:13.982928761 +0000 UTC m=+2501.056181124" Jul 02 23:43:15.275156 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:15.275125 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d6kx2_759fdb13-b1f4-4476-8a9d-2b52e33105aa/dns/0.log" Jul 02 23:43:15.295198 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:15.295169 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d6kx2_759fdb13-b1f4-4476-8a9d-2b52e33105aa/kube-rbac-proxy/0.log" Jul 02 23:43:15.422539 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:15.422508 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4vks2_12a6f254-8a6d-4f61-a3c9-220fb8d9f1fd/dns-node-resolver/0.log" Jul 02 23:43:15.967717 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:15.967687 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-55fbb5d77-ff26t_9246c550-93af-4a26-9efb-2af46970ccdb/registry/0.log" Jul 02 23:43:15.986731 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:15.986704 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gfdss_ed9789e7-960c-4bb6-b65d-fd51ae63a29e/node-ca/0.log" Jul 02 23:43:17.070676 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:17.070645 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-99946bbb8-tg2t5_d9a1431f-25ad-48b5-b658-86f757f2f5a1/kube-auth-proxy/0.log" Jul 02 23:43:17.904611 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:17.904581 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kdqh7_54bffe79-6186-4756-a955-f21219245e06/serve-healthcheck-canary/0.log" Jul 02 23:43:18.498911 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:18.498878 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jkqkn_c3288985-9f79-4260-8ab0-b35085c5c823/kube-rbac-proxy/0.log" Jul 02 23:43:18.517358 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:18.517332 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jkqkn_c3288985-9f79-4260-8ab0-b35085c5c823/exporter/0.log" Jul 02 23:43:18.535528 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:18.535506 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jkqkn_c3288985-9f79-4260-8ab0-b35085c5c823/extractor/0.log" Jul 02 23:43:19.984171 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:19.984145 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dblpz/perf-node-gather-daemonset-5d5wr" Jul 02 23:43:20.930984 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:20.930948 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-4xvtf_68e8c8db-a45b-47f0-8918-1ece0183f62e/manager/0.log" Jul 02 23:43:21.084972 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:21.084935 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-676b9dcdc-q5zgq_3e845ee6-2bd5-4ef2-bbc0-445166acc977/manager/0.log" Jul 02 23:43:21.103342 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:21.103312 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-676b9dcdc-q5zgq_3e845ee6-2bd5-4ef2-bbc0-445166acc977/copy-manifests/0.log" Jul 02 23:43:22.563631 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:22.563543 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7f69f5c6b6-j42gx_efe92fb8-f483-4e1c-b64d-1711720f5bc2/manager/0.log" Jul 02 23:43:23.610761 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:23.610729 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-99946bbb8-tg2t5_d9a1431f-25ad-48b5-b658-86f757f2f5a1/kube-auth-proxy/0.log" Jul 02 23:43:25.043504 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:25.043471 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-99946bbb8-tg2t5_d9a1431f-25ad-48b5-b658-86f757f2f5a1/kube-auth-proxy/0.log" Jul 02 23:43:29.054366 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:29.054262 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xr5tz_ce7afefc-665b-46ab-9895-38facdf8467f/kube-multus-additional-cni-plugins/0.log" Jul 02 23:43:29.072337 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:29.072309 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xr5tz_ce7afefc-665b-46ab-9895-38facdf8467f/egress-router-binary-copy/0.log" Jul 02 23:43:29.105411 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:29.105388 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xr5tz_ce7afefc-665b-46ab-9895-38facdf8467f/cni-plugins/0.log" Jul 02 23:43:29.125339 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:29.125307 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xr5tz_ce7afefc-665b-46ab-9895-38facdf8467f/bond-cni-plugin/0.log" Jul 02 23:43:29.142145 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:29.142126 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xr5tz_ce7afefc-665b-46ab-9895-38facdf8467f/routeoverride-cni/0.log" Jul 02 23:43:29.159842 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:29.159825 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xr5tz_ce7afefc-665b-46ab-9895-38facdf8467f/whereabouts-cni-bincopy/0.log" Jul 02 23:43:29.178129 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:29.178110 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xr5tz_ce7afefc-665b-46ab-9895-38facdf8467f/whereabouts-cni/0.log" Jul 02 23:43:29.266654 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:29.266628 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-k5dh5_9f933ee1-d50b-4391-9881-d002c7ef4ab9/kube-multus/0.log" Jul 02 23:43:29.322560 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:29.322480 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mm52v_bad930f0-e377-445a-88a5-7e81b222ce9b/network-metrics-daemon/0.log" Jul 02 23:43:29.338477 ip-10-0-136-71 kubenswrapper[2568]: I0702 23:43:29.338452 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mm52v_bad930f0-e377-445a-88a5-7e81b222ce9b/kube-rbac-proxy/0.log"