Jul 15 16:57:00.524095 ip-10-0-135-132 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 16:57:00.524107 ip-10-0-135-132 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 16:57:00.524116 ip-10-0-135-132 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 16:57:00.524412 ip-10-0-135-132 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 16:57:12.058370 ip-10-0-135-132 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 15 16:57:12.058388 ip-10-0-135-132 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ad3aaf54d72f477282fe444952d9c9a5 -- Jul 15 16:59:09.663125 ip-10-0-135-132 systemd[1]: Starting Kubernetes Kubelet... Jul 15 16:59:09.971109 ip-10-0-135-132 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:59:09.971109 ip-10-0-135-132 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 16:59:09.971109 ip-10-0-135-132 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:59:09.971109 ip-10-0-135-132 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 16:59:09.971109 ip-10-0-135-132 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:59:09.972697 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.972614 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 16:59:09.977634 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977615 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:59:09.977634 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977631 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:59:09.977634 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977635 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:59:09.977634 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977638 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:59:09.977634 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977641 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977644 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977647 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977649 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977652 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977654 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977657 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977659 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977661 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977664 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977666 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977669 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977672 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977675 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977677 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977681 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977683 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977686 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977688 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977691 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:59:09.977815 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977693 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977696 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977698 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977700 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977703 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977705 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977708 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977712 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977716 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977719 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977721 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977724 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977727 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977729 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977732 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977735 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977737 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977740 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977742 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:59:09.978346 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977745 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977747 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977750 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977752 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977755 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977757 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977759 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977762 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977765 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977768 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977771 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977773 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977776 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977778 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977780 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977783 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977785 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977787 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977790 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977792 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:59:09.978918 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977795 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977797 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977799 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977802 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977805 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977807 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977810 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977813 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977815 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977818 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977820 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977823 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977825 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977828 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977830 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977848 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977851 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977853 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977856 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977859 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:59:09.979405 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977862 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977864 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977868 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977872 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977875 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977878 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977881 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977884 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977887 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977890 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977892 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977895 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977899 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977902 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977904 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.977907 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978573 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978580 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978583 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:59:09.979916 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978586 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978589 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978591 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978595 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978599 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978602 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978604 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978607 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978610 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978612 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978615 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978617 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978620 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978622 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978625 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978627 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978630 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978632 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978635 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:59:09.980384 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978639 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978642 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978645 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978648 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978651 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978653 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978656 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978658 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978661 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978663 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978666 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978669 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978671 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978674 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978677 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978679 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978682 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978684 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978687 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978689 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:59:09.980933 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978692 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978694 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978696 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978699 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978701 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978704 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978706 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978710 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978714 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978717 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978720 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978723 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978726 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978728 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978731 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978733 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978736 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978738 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978741 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:59:09.981424 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978743 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978746 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978748 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978751 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978753 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978756 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978758 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978760 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978763 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978766 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978768 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978770 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978773 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978775 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978778 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978780 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978782 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978785 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978788 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978791 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:59:09.981905 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978793 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978795 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978798 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978801 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978803 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978806 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978809 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978811 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978813 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978816 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978818 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978821 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978823 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978826 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978828 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978846 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978849 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.978852 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979194 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979198 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:59:09.982390 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979201 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979204 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979207 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979209 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979212 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979215 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979217 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979220 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979223 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979226 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979228 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979230 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979233 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979235 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979238 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979241 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979243 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979245 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979248 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979250 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:59:09.982874 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979253 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979255 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979258 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979261 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979263 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979265 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979268 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979270 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979273 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979275 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979278 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979280 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979282 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979285 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979287 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979289 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979292 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979294 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979296 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979299 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:59:09.983359 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979301 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979305 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979309 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979312 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979315 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979317 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979319 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979322 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979325 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979327 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979329 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979332 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979335 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979337 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979339 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979342 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979344 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979347 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979349 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:59:09.983856 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979351 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979354 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979359 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979361 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979363 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979366 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979368 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979372 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979375 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979377 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979380 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979382 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979385 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979387 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979389 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979392 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979394 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979397 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979401 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:59:09.984336 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979404 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979407 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979410 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979414 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979418 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979421 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979424 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979427 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979430 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979432 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979435 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979438 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979440 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979443 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979445 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979449 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979451 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979454 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.979456 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979535 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 16:59:09.984804 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979545 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979555 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979560 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979566 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979570 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979574 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979577 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979580 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979583 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979586 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979590 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979592 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979595 2562 flags.go:64] FLAG: --cgroup-root="" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979598 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979601 2562 flags.go:64] FLAG: --client-ca-file="" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979604 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979607 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979610 2562 flags.go:64] FLAG: --cluster-domain="" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979613 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979616 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979619 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979622 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979625 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 16:59:09.985310 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979628 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979631 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979634 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979637 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979641 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979644 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979647 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979649 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979653 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979656 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979659 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979661 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979664 2562 flags.go:64] FLAG: --enable-server="true" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979667 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979672 2562 flags.go:64] FLAG: --event-burst="100" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979675 2562 flags.go:64] FLAG: --event-qps="50" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979677 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979680 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979683 2562 flags.go:64] FLAG: --eviction-hard="" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979687 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979689 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979692 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979695 2562 flags.go:64] FLAG: --eviction-soft="" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979698 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979700 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 16:59:09.985879 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979707 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979712 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979715 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979718 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979720 2562 flags.go:64] FLAG: --feature-gates="" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979724 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979727 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979729 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979732 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979735 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979738 2562 flags.go:64] FLAG: --help="false" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979743 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-135-132.ec2.internal" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979746 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979749 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979751 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979755 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979758 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979760 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979763 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979765 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979768 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979771 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979774 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979776 2562 flags.go:64] FLAG: --kube-reserved="" Jul 15 16:59:09.986557 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979779 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979782 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979784 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979787 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979790 2562 flags.go:64] FLAG: --lock-file="" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979792 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979795 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979798 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979802 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979807 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979810 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979812 2562 flags.go:64] FLAG: --logging-format="text" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979815 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979818 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979821 2562 flags.go:64] FLAG: --manifest-url="" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979823 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979827 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979830 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979851 2562 flags.go:64] FLAG: --max-pods="110" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979855 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979858 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979861 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979863 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979866 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979869 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 16:59:09.987159 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979872 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979879 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979882 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979885 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979888 2562 flags.go:64] FLAG: --pod-cidr="" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979890 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979895 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979898 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979901 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979904 2562 flags.go:64] FLAG: --port="10250" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979907 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979910 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e1e74c3a1d98d472" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979913 2562 flags.go:64] FLAG: --qos-reserved="" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979916 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979919 2562 flags.go:64] FLAG: --register-node="true" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979922 2562 flags.go:64] FLAG: --register-with-taints="" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979925 2562 flags.go:64] FLAG: --registry-burst="10" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979936 2562 flags.go:64] FLAG: --registry-qps="5" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979939 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979942 2562 flags.go:64] FLAG: --reserved-memory="" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979949 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979952 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979955 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979958 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979960 2562 flags.go:64] FLAG: --runonce="false" Jul 15 16:59:09.987794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979963 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979967 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979970 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979973 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979975 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979978 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979981 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979984 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979986 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979989 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979992 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979994 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.979997 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980000 2562 flags.go:64] FLAG: --system-cgroups="" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980003 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980008 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980010 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980013 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980066 2562 flags.go:64] FLAG: --tls-min-version="" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980069 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980072 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980075 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980078 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980081 2562 flags.go:64] FLAG: --v="2" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980086 2562 flags.go:64] FLAG: --version="false" Jul 15 16:59:09.988427 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980091 2562 flags.go:64] FLAG: --vmodule="" Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980094 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.980097 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980209 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980213 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980215 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980218 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980221 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980225 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980229 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980233 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980236 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980239 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980243 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980247 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980250 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980253 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980256 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980259 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:59:09.989063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980261 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980263 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980266 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980268 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980271 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980274 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980276 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980279 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980281 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980284 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980286 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980289 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980292 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980295 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980298 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980301 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980303 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980306 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980308 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980310 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:59:09.989525 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980313 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980318 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980321 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980324 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980327 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980330 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980333 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980336 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980338 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980341 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980344 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980347 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980349 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980352 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980354 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980357 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980359 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980362 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980365 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980367 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:59:09.990135 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980369 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980372 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980374 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980377 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980382 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980384 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980387 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980390 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980392 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980395 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980397 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980400 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980402 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980406 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980408 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980411 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980413 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980416 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980419 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980421 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:59:09.990632 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980423 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980426 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980429 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980431 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980434 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980436 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980439 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980441 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980444 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980446 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980448 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980451 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980453 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980455 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980458 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980460 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980471 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980474 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980477 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980479 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:59:09.991133 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980482 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980485 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.980487 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.981032 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.987655 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.987668 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.988469 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.988476 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988549 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988554 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988557 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988560 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988563 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988566 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:59:09.991675 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988568 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988571 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988574 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988577 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988580 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988582 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988585 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988587 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988589 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988592 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988594 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988597 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988599 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988602 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988604 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988607 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988610 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988612 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988615 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988617 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:59:09.992048 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988620 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988624 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988628 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988632 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988635 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988638 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988641 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988644 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988646 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988649 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988652 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988654 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988657 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988659 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988661 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988664 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988666 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988669 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988671 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:59:09.992547 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988675 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988679 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988681 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988684 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988687 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988689 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988692 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988695 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988697 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988700 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988702 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988705 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988707 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988710 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988713 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988715 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988718 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988722 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988726 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:59:09.993041 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988729 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988731 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988734 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988737 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988739 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988742 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988744 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988747 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988749 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988752 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988755 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988757 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988759 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988762 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988764 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988767 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988769 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988772 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988774 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988777 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:59:09.993501 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988779 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988782 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988784 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988787 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988789 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988791 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988794 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988796 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988799 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988802 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988804 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988807 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988810 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988812 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988814 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:59:09.994076 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.988819 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988985 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988991 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988994 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.988996 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989000 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989003 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989006 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989008 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989011 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989013 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989016 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989019 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989021 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989023 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989026 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989028 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989031 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989034 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989036 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:59:09.994449 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989038 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989041 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989043 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989046 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989048 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989050 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989054 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989056 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989059 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989067 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989070 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989072 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989075 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989077 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989080 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989082 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989084 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989087 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989089 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989097 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:59:09.995011 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989099 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989102 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989104 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989107 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989109 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989112 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989114 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989117 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989119 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989121 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989124 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989126 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989128 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989130 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989133 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989136 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989138 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989141 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989143 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989145 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:59:09.995500 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989148 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989150 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989161 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989164 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989167 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989170 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989172 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989175 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989178 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989180 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989183 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989185 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989187 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989190 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989192 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989194 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989197 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989199 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989201 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:59:09.996034 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989204 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989206 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989208 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989211 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989213 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989215 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989218 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989220 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989223 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989225 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989227 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989229 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989232 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989234 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989238 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989241 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989249 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989252 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989254 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:59:09.996572 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989257 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:59:09.997216 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:09.989260 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:59:09.997216 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.989265 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:59:09.997216 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.989378 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 16:59:09.997216 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.991335 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 16:59:09.997216 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.992482 2562 server.go:1019] "Starting client certificate rotation" Jul 15 16:59:09.997216 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.992584 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 16:59:09.997216 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.992617 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 16:59:09.997216 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.994580 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 16:59:09.998083 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:09.998066 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 16:59:10.005186 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.005168 2562 log.go:25] "Validated CRI v1 runtime API" Jul 15 16:59:10.007451 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.007437 2562 log.go:25] "Validated CRI v1 image API" Jul 15 16:59:10.008167 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.008153 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 16:59:10.013046 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.013028 2562 fs.go:135] Filesystem UUIDs: map[0472f209-7fe2-4ccf-bb00-eae5fb423bb4:/dev/nvme0n1p4 058535c9-fe91-45ec-93f8-e0bc6f7b2ecd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 15 16:59:10.013103 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.013046 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 16:59:10.017792 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.017684 2562 manager.go:216] Machine: {Timestamp:2026-07-15 16:59:10.016563477 +0000 UTC m=+0.258558694 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3071942 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec281df6ed4adf81faab807c413a58e3 SystemUUID:ec281df6-ed4a-df81-faab-807c413a58e3 BootID:ad3aaf54-d72f-4772-82fe-444952d9c9a5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e2:a1:ef:9c:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e2:a1:ef:9c:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:d8:5f:27:d1:ab Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 16:59:10.017792 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.017787 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 16:59:10.017920 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.017874 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 16:59:10.020232 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.020208 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 16:59:10.020354 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.020236 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-132.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 16:59:10.020951 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.020941 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 16:59:10.020989 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.020954 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 16:59:10.020989 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.020961 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 16:59:10.021522 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.021513 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 16:59:10.022461 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.022450 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 16:59:10.024933 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.024918 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 16:59:10.026240 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.026229 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 15 16:59:10.026362 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.026352 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 16:59:10.026398 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.026383 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 15 16:59:10.026398 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.026395 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 16:59:10.026452 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.026428 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 16:59:10.026452 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.026437 2562 kubelet.go:398] "Adding apiserver pod source" Jul 15 16:59:10.026452 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.026449 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 16:59:10.026938 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.026908 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 16:59:10.026997 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.026951 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 16:59:10.028864 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.028849 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 16:59:10.030258 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030242 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 16:59:10.030297 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030275 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 16:59:10.030915 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030904 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 16:59:10.030947 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030920 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 16:59:10.030947 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030926 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 16:59:10.030947 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030931 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 16:59:10.031030 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030941 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 16:59:10.031030 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030960 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 16:59:10.031030 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030968 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 16:59:10.031030 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030973 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 16:59:10.031030 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030979 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 16:59:10.031030 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030984 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 16:59:10.031030 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.030997 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 16:59:10.031030 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.031007 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 16:59:10.031689 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.031680 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 16:59:10.031689 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.031689 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 16:59:10.035785 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.035769 2562 server.go:1268] "Started kubelet" Jul 15 16:59:10.036018 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.035881 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 16:59:10.036018 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.035905 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 16:59:10.036018 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.035952 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 16:59:10.036652 ip-10-0-135-132 systemd[1]: Started Kubernetes Kubelet. Jul 15 16:59:10.037691 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.037641 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 15 16:59:10.039626 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.039606 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 16:59:10.041607 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.041582 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 16:59:10.043001 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.042983 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 16:59:10.043083 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.043028 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 16:59:10.043192 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.043175 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:10.043609 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.043567 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 16:59:10.043976 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.043960 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 16:59:10.043976 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.043977 2562 factory.go:55] Registering systemd factory Jul 15 16:59:10.044107 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.043986 2562 factory.go:223] Registration of the systemd container factory successfully Jul 15 16:59:10.044107 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.044046 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 16:59:10.044107 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.044057 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 16:59:10.044240 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.044170 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-132.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 16:59:10.044240 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.044220 2562 factory.go:153] Registering CRI-O factory Jul 15 16:59:10.044240 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.044232 2562 factory.go:223] Registration of the crio container factory successfully Jul 15 16:59:10.044332 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.044256 2562 factory.go:103] Registering Raw factory Jul 15 16:59:10.044332 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.044273 2562 manager.go:1196] Started watching for new ooms in manager Jul 15 16:59:10.044409 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.044341 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 16:59:10.044493 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.044404 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-132.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 16:59:10.044530 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.044498 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 15 16:59:10.044530 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.044510 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 15 16:59:10.044868 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.044041 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-132.ec2.internal.18c2851a374f2887 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-132.ec2.internal,UID:ip-10-0-135-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-132.ec2.internal,},FirstTimestamp:2026-07-15 16:59:10.035716231 +0000 UTC m=+0.277711449,LastTimestamp:2026-07-15 16:59:10.035716231 +0000 UTC m=+0.277711449,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-132.ec2.internal,}" Jul 15 16:59:10.045053 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.045041 2562 manager.go:349] Starting recovery of all containers Jul 15 16:59:10.049875 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.049827 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 16:59:10.050025 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.049986 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-132.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 16:59:10.059213 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.059098 2562 manager.go:354] Recovery completed Jul 15 16:59:10.063382 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.063365 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:59:10.065147 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.065121 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:59:10.065222 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.065164 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:59:10.065222 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.065178 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:59:10.065688 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.065675 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 16:59:10.065688 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.065688 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 16:59:10.065793 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.065707 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 15 16:59:10.065793 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.065717 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:59:10.066416 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.066400 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:59:10.066479 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.066429 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:59:10.066479 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.066445 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:59:10.067718 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.067654 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-132.ec2.internal.18c2851a39103cc3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-132.ec2.internal,UID:ip-10-0-135-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-132.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-132.ec2.internal,},FirstTimestamp:2026-07-15 16:59:10.065147075 +0000 UTC m=+0.307142302,LastTimestamp:2026-07-15 16:59:10.065147075 +0000 UTC m=+0.307142302,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-132.ec2.internal,}" Jul 15 16:59:10.067945 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.067927 2562 policy_none.go:49] "None policy: Start" Jul 15 16:59:10.067945 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.067945 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 16:59:10.068063 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.067955 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 16:59:10.068432 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.068416 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p6jfg" Jul 15 16:59:10.069947 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.069936 2562 policy_none.go:47] "Start" Jul 15 16:59:10.077401 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.077345 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-132.ec2.internal.18c2851a39109a52 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-132.ec2.internal,UID:ip-10-0-135-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-132.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-132.ec2.internal,},FirstTimestamp:2026-07-15 16:59:10.065171026 +0000 UTC m=+0.307166247,LastTimestamp:2026-07-15 16:59:10.065171026 +0000 UTC m=+0.307166247,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-132.ec2.internal,}" Jul 15 16:59:10.079743 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.079688 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-132.ec2.internal.18c2851a3910cbbd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-132.ec2.internal,UID:ip-10-0-135-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-132.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-132.ec2.internal,},FirstTimestamp:2026-07-15 16:59:10.065183677 +0000 UTC m=+0.307178899,LastTimestamp:2026-07-15 16:59:10.065183677 +0000 UTC m=+0.307178899,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-132.ec2.internal,}" Jul 15 16:59:10.083418 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.083399 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p6jfg" Jul 15 16:59:10.087609 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.087532 2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-132.ec2.internal.18c2851a39103cc3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-132.ec2.internal.18c2851a39103cc3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-132.ec2.internal,UID:ip-10-0-135-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-132.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-132.ec2.internal,},FirstTimestamp:2026-07-15 16:59:10.065147075 +0000 UTC m=+0.307142302,LastTimestamp:2026-07-15 16:59:10.066415194 +0000 UTC m=+0.308410412,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-132.ec2.internal,}" Jul 15 16:59:10.105390 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.105375 2562 manager.go:337] "Starting Device Plugin manager" Jul 15 16:59:10.105477 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.105415 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 16:59:10.105477 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.105428 2562 server.go:87] "Starting device plugin registration server" Jul 15 16:59:10.105664 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.105651 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 16:59:10.105726 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.105665 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 16:59:10.105774 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.105749 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 16:59:10.105854 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.105819 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 16:59:10.105911 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.105830 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 16:59:10.107481 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.107460 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 16:59:10.107539 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.107517 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:10.206279 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.206258 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:59:10.208459 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.207033 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:59:10.208459 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.207065 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:59:10.208459 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.207078 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:59:10.208459 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.207100 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.214335 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.214321 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.214411 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.214340 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-132.ec2.internal\": node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:10.243486 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.243428 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 16:59:10.243950 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.243923 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:10.244941 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.244928 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 16:59:10.245007 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.244966 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 16:59:10.245042 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.245012 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 16:59:10.245073 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.245042 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 16:59:10.250345 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.250322 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:59:10.344849 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.344811 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:10.345909 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.345892 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal"] Jul 15 16:59:10.345955 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.345951 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:59:10.346768 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.346753 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:59:10.346810 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.346782 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:59:10.346810 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.346793 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:59:10.349158 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.349147 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:59:10.349305 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.349288 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.349362 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.349327 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:59:10.349930 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.349910 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:59:10.350007 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.349930 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:59:10.350007 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.349956 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:59:10.350007 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.349966 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:59:10.350007 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.349955 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:59:10.350128 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.350023 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:59:10.352345 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.352332 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.352394 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.352386 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:59:10.353065 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.353047 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:59:10.353156 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.353075 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:59:10.353156 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.353087 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:59:10.376421 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.376401 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-132.ec2.internal\" not found" node="ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.380650 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.380633 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-132.ec2.internal\" not found" node="ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.445738 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.445716 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:10.446854 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.446824 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b41c58adddba08521de05a44de7576c8-config\") pod \"kube-apiserver-proxy-ip-10-0-135-132.ec2.internal\" (UID: \"b41c58adddba08521de05a44de7576c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.446902 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.446863 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1176b310d9fb52e4fefed71966d7b67a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"1176b310d9fb52e4fefed71966d7b67a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.446902 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.446881 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1176b310d9fb52e4fefed71966d7b67a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"1176b310d9fb52e4fefed71966d7b67a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.546887 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.546807 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:10.547925 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.547910 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1176b310d9fb52e4fefed71966d7b67a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"1176b310d9fb52e4fefed71966d7b67a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.547975 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.547933 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1176b310d9fb52e4fefed71966d7b67a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"1176b310d9fb52e4fefed71966d7b67a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.547975 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.547953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b41c58adddba08521de05a44de7576c8-config\") pod \"kube-apiserver-proxy-ip-10-0-135-132.ec2.internal\" (UID: \"b41c58adddba08521de05a44de7576c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.548034 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.547993 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b41c58adddba08521de05a44de7576c8-config\") pod \"kube-apiserver-proxy-ip-10-0-135-132.ec2.internal\" (UID: \"b41c58adddba08521de05a44de7576c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.548077 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.548025 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1176b310d9fb52e4fefed71966d7b67a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"1176b310d9fb52e4fefed71966d7b67a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.548077 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.548043 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1176b310d9fb52e4fefed71966d7b67a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal\" (UID: \"1176b310d9fb52e4fefed71966d7b67a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.647285 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.647245 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:10.679510 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.679483 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.683176 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.683131 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" Jul 15 16:59:10.748059 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.748032 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:10.848496 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.848434 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:10.949011 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:10.948986 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:10.992514 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.992490 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 16:59:10.993003 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:10.992618 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 16:59:11.049489 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:11.049467 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:11.089080 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.089037 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 16:54:10 +0000 UTC" deadline="2027-12-13 07:24:45.45657543 +0000 UTC" Jul 15 16:59:11.089080 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.089074 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12374h25m34.367504191s" Jul 15 16:59:11.110466 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.110405 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:59:11.111496 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.111478 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:59:11.111586 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.111513 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:59:11.111586 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.111523 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:59:11.149820 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:11.149791 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-132.ec2.internal\" not found" Jul 15 16:59:11.192060 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.192037 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:59:11.245963 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.245936 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" Jul 15 16:59:11.259938 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.259918 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 16:59:11.261366 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.261348 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" Jul 15 16:59:11.275045 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:11.274687 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41c58adddba08521de05a44de7576c8.slice/crio-d4165b1304b9458ba269d08dbf32ec3dc4f095719f30bb62c6d7142fa28260a6 WatchSource:0}: Error finding container d4165b1304b9458ba269d08dbf32ec3dc4f095719f30bb62c6d7142fa28260a6: Status 404 returned error can't find the container with id d4165b1304b9458ba269d08dbf32ec3dc4f095719f30bb62c6d7142fa28260a6 Jul 15 16:59:11.279057 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.278959 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 16:59:11.279320 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.279307 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 16:59:11.428731 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.428642 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:59:11.654988 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:11.654960 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:59:11.874523 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:11.874488 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1176b310d9fb52e4fefed71966d7b67a.slice/crio-aefd244820935d0aaa9df301738ada05b7405d59c740e247050e63a6abe2ce3a WatchSource:0}: Error finding container aefd244820935d0aaa9df301738ada05b7405d59c740e247050e63a6abe2ce3a: Status 404 returned error can't find the container with id aefd244820935d0aaa9df301738ada05b7405d59c740e247050e63a6abe2ce3a Jul 15 16:59:12.028174 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.028138 2562 apiserver.go:52] "Watching apiserver" Jul 15 16:59:12.032994 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.032959 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 16:59:12.033266 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.033244 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-l46zz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-vclsh","kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr","openshift-cluster-node-tuning-operator/tuned-vvw4b"] Jul 15 16:59:12.035733 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.035710 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l46zz" Jul 15 16:59:12.038297 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.038270 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.039199 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.039176 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 16:59:12.039342 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.039176 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kdwbd\"" Jul 15 16:59:12.040488 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.040465 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 16:59:12.040595 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.040549 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 16:59:12.040812 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.040796 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.043405 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.043246 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.044089 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.044071 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 16:59:12.044504 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.044483 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 16:59:12.049930 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.049909 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 16:59:12.050043 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.049932 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 16:59:12.050129 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.050102 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-92tpp\"" Jul 15 16:59:12.051174 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.051154 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:59:12.051439 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.051309 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 16:59:12.051439 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.051327 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 16:59:12.051439 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.051423 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 16:59:12.051823 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.051793 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fcnm4\"" Jul 15 16:59:12.052059 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.052032 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 16:59:12.052806 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.052177 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 16:59:12.052806 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.052233 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 16:59:12.052806 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.052327 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wtgrq\"" Jul 15 16:59:12.052806 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.052427 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 16:59:12.053191 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.053172 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 16:59:12.056612 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-etc-selinux\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.056719 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056621 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-host\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.056719 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056642 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/435df3eb-d81b-471e-b851-dbb91a35dbce-host\") pod \"node-ca-l46zz\" (UID: \"435df3eb-d81b-471e-b851-dbb91a35dbce\") " pod="openshift-image-registry/node-ca-l46zz" Jul 15 16:59:12.056719 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056690 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-run-systemd\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.056919 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056749 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-cni-bin\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.056919 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056785 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-cni-netd\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.056919 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056810 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-systemd\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.056919 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4srx\" (UniqueName: \"kubernetes.io/projected/435df3eb-d81b-471e-b851-dbb91a35dbce-kube-api-access-j4srx\") pod \"node-ca-l46zz\" (UID: \"435df3eb-d81b-471e-b851-dbb91a35dbce\") " pod="openshift-image-registry/node-ca-l46zz" Jul 15 16:59:12.056919 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056898 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-systemd-units\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.057177 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056925 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-socket-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.057177 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056945 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-sys-fs\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.057177 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056960 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-kubernetes\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.057177 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.056974 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-run\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.057177 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-lib-modules\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.057177 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057040 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9ab826df-bc9b-42a8-bb4a-989adbec2594-ovnkube-script-lib\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.057177 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057075 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.057177 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-registration-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.057177 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057135 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htdjm\" (UniqueName: \"kubernetes.io/projected/9a0bd977-5444-4eb2-b422-f45950511b16-kube-api-access-htdjm\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.057177 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057160 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-run-openvswitch\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057184 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-modprobe-d\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057221 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-sysctl-d\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057270 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-sys\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057295 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-tuned\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057321 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-log-socket\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057356 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9ab826df-bc9b-42a8-bb4a-989adbec2594-ovn-node-metrics-cert\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057393 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-device-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-sysctl-conf\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057458 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-var-lib-kubelet\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057483 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-run-ovn\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057514 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-node-log\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.057621 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057556 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5s94\" (UniqueName: \"kubernetes.io/projected/9ab826df-bc9b-42a8-bb4a-989adbec2594-kube-api-access-b5s94\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057612 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-sysconfig\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-tmp\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057701 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr5hz\" (UniqueName: \"kubernetes.io/projected/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-kube-api-access-gr5hz\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057732 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/435df3eb-d81b-471e-b851-dbb91a35dbce-serviceca\") pod \"node-ca-l46zz\" (UID: \"435df3eb-d81b-471e-b851-dbb91a35dbce\") " pod="openshift-image-registry/node-ca-l46zz" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057761 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-kubelet\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057782 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-slash\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057807 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-run-netns\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057832 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-etc-openvswitch\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057904 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9ab826df-bc9b-42a8-bb4a-989adbec2594-ovnkube-config\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057932 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-var-lib-openvswitch\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057957 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-run-ovn-kubernetes\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.057986 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.058244 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.058004 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9ab826df-bc9b-42a8-bb4a-989adbec2594-env-overrides\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.067915 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.067882 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 16:59:12.119515 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.119472 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sd27v" Jul 15 16:59:12.159098 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159011 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-device-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.159098 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-sysctl-conf\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.159321 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159145 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-device-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.159321 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159196 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-var-lib-kubelet\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.159321 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159235 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-run-ovn\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159321 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159264 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-node-log\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159321 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159199 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-sysctl-conf\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.159321 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5s94\" (UniqueName: \"kubernetes.io/projected/9ab826df-bc9b-42a8-bb4a-989adbec2594-kube-api-access-b5s94\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159321 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-sysconfig\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159334 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-var-lib-kubelet\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159342 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-tmp\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159368 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-node-log\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159366 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr5hz\" (UniqueName: \"kubernetes.io/projected/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-kube-api-access-gr5hz\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/435df3eb-d81b-471e-b851-dbb91a35dbce-serviceca\") pod \"node-ca-l46zz\" (UID: \"435df3eb-d81b-471e-b851-dbb91a35dbce\") " pod="openshift-image-registry/node-ca-l46zz" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-kubelet\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-slash\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-run-netns\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-etc-openvswitch\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159517 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-sysconfig\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9ab826df-bc9b-42a8-bb4a-989adbec2594-ovnkube-config\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159583 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-var-lib-openvswitch\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-run-ovn\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159614 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-run-ovn-kubernetes\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159645 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159653 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-kubelet\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.159668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159673 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9ab826df-bc9b-42a8-bb4a-989adbec2594-env-overrides\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159696 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-slash\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159702 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-etc-selinux\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159756 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-run-netns\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159781 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159800 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-etc-openvswitch\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159802 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159822 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-run-ovn-kubernetes\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159840 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-etc-selinux\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159851 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-var-lib-openvswitch\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159863 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-host\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159961 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-host\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159970 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/435df3eb-d81b-471e-b851-dbb91a35dbce-host\") pod \"node-ca-l46zz\" (UID: \"435df3eb-d81b-471e-b851-dbb91a35dbce\") " pod="openshift-image-registry/node-ca-l46zz" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.159998 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-run-systemd\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160016 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-cni-bin\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-cni-netd\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160052 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-systemd\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4srx\" (UniqueName: \"kubernetes.io/projected/435df3eb-d81b-471e-b851-dbb91a35dbce-kube-api-access-j4srx\") pod \"node-ca-l46zz\" (UID: \"435df3eb-d81b-471e-b851-dbb91a35dbce\") " pod="openshift-image-registry/node-ca-l46zz" Jul 15 16:59:12.160453 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160096 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-cni-bin\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160103 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-systemd-units\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160106 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/435df3eb-d81b-471e-b851-dbb91a35dbce-host\") pod \"node-ca-l46zz\" (UID: \"435df3eb-d81b-471e-b851-dbb91a35dbce\") " pod="openshift-image-registry/node-ca-l46zz" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160137 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-socket-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160146 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-systemd-units\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160156 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-run-systemd\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160168 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-sys-fs\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160168 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/435df3eb-d81b-471e-b851-dbb91a35dbce-serviceca\") pod \"node-ca-l46zz\" (UID: \"435df3eb-d81b-471e-b851-dbb91a35dbce\") " pod="openshift-image-registry/node-ca-l46zz" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160193 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-host-cni-netd\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160199 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9ab826df-bc9b-42a8-bb4a-989adbec2594-env-overrides\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160206 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9ab826df-bc9b-42a8-bb4a-989adbec2594-ovnkube-config\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160195 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-kubernetes\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160240 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-kubernetes\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160245 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-run\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160211 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-systemd\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160270 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-sys-fs\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160276 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-lib-modules\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160289 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-run\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160302 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-socket-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160312 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9ab826df-bc9b-42a8-bb4a-989adbec2594-ovnkube-script-lib\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160336 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160352 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-registration-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160372 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdjm\" (UniqueName: \"kubernetes.io/projected/9a0bd977-5444-4eb2-b422-f45950511b16-kube-api-access-htdjm\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160397 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-lib-modules\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160415 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160445 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-run-openvswitch\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a0bd977-5444-4eb2-b422-f45950511b16-registration-dir\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-modprobe-d\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-sysctl-d\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160508 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-run-openvswitch\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160516 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-sys\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160541 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-tuned\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160565 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-log-socket\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160591 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9ab826df-bc9b-42a8-bb4a-989adbec2594-ovn-node-metrics-cert\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160609 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-sysctl-d\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.161893 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160611 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-modprobe-d\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.162549 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160664 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9ab826df-bc9b-42a8-bb4a-989adbec2594-log-socket\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.162549 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160675 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-sys\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.162549 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.160904 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9ab826df-bc9b-42a8-bb4a-989adbec2594-ovnkube-script-lib\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.163633 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.163550 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-tmp\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.163633 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.163595 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-etc-tuned\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.163782 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.163737 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9ab826df-bc9b-42a8-bb4a-989adbec2594-ovn-node-metrics-cert\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.177579 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.177552 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4srx\" (UniqueName: \"kubernetes.io/projected/435df3eb-d81b-471e-b851-dbb91a35dbce-kube-api-access-j4srx\") pod \"node-ca-l46zz\" (UID: \"435df3eb-d81b-471e-b851-dbb91a35dbce\") " pod="openshift-image-registry/node-ca-l46zz" Jul 15 16:59:12.184175 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.184146 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr5hz\" (UniqueName: \"kubernetes.io/projected/c5b4654a-99b4-4147-b754-4ce04a4a2ac0-kube-api-access-gr5hz\") pod \"tuned-vvw4b\" (UID: \"c5b4654a-99b4-4147-b754-4ce04a4a2ac0\") " pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.184282 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.184195 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htdjm\" (UniqueName: \"kubernetes.io/projected/9a0bd977-5444-4eb2-b422-f45950511b16-kube-api-access-htdjm\") pod \"aws-ebs-csi-driver-node-dqnlr\" (UID: \"9a0bd977-5444-4eb2-b422-f45950511b16\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.184282 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.184209 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5s94\" (UniqueName: \"kubernetes.io/projected/9ab826df-bc9b-42a8-bb4a-989adbec2594-kube-api-access-b5s94\") pod \"ovnkube-node-vclsh\" (UID: \"9ab826df-bc9b-42a8-bb4a-989adbec2594\") " pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.249420 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.249356 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" event={"ID":"b41c58adddba08521de05a44de7576c8","Type":"ContainerStarted","Data":"d4165b1304b9458ba269d08dbf32ec3dc4f095719f30bb62c6d7142fa28260a6"} Jul 15 16:59:12.250094 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.250072 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:59:12.250511 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.250488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" event={"ID":"1176b310d9fb52e4fefed71966d7b67a","Type":"ContainerStarted","Data":"aefd244820935d0aaa9df301738ada05b7405d59c740e247050e63a6abe2ce3a"} Jul 15 16:59:12.347939 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.347894 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l46zz" Jul 15 16:59:12.355927 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.355897 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:12.366157 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.366135 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" Jul 15 16:59:12.370047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:12.370028 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" Jul 15 16:59:12.961695 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:12.961664 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a0bd977_5444_4eb2_b422_f45950511b16.slice/crio-b19a684b148a27a41bf4ed6ca16b63fa2d95c6874f864b6365149caf08e012f5 WatchSource:0}: Error finding container b19a684b148a27a41bf4ed6ca16b63fa2d95c6874f864b6365149caf08e012f5: Status 404 returned error can't find the container with id b19a684b148a27a41bf4ed6ca16b63fa2d95c6874f864b6365149caf08e012f5 Jul 15 16:59:12.963303 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:12.963142 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b4654a_99b4_4147_b754_4ce04a4a2ac0.slice/crio-2945347e85daa81a116b319a340dad72e86824cc493f79b3bbbf01360888d54e WatchSource:0}: Error finding container 2945347e85daa81a116b319a340dad72e86824cc493f79b3bbbf01360888d54e: Status 404 returned error can't find the container with id 2945347e85daa81a116b319a340dad72e86824cc493f79b3bbbf01360888d54e Jul 15 16:59:12.964063 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:12.964046 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod435df3eb_d81b_471e_b851_dbb91a35dbce.slice/crio-909928ee77a6cc6d3b6c013f545c1423a4b0f6ac8044bd318378d88dbb52101a WatchSource:0}: Error finding container 909928ee77a6cc6d3b6c013f545c1423a4b0f6ac8044bd318378d88dbb52101a: Status 404 returned error can't find the container with id 909928ee77a6cc6d3b6c013f545c1423a4b0f6ac8044bd318378d88dbb52101a Jul 15 16:59:12.964493 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:12.964429 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab826df_bc9b_42a8_bb4a_989adbec2594.slice/crio-8f66227489c9cde7b764ac25ee24a5042b50e68faf1c37ebc934f8314636da89 WatchSource:0}: Error finding container 8f66227489c9cde7b764ac25ee24a5042b50e68faf1c37ebc934f8314636da89: Status 404 returned error can't find the container with id 8f66227489c9cde7b764ac25ee24a5042b50e68faf1c37ebc934f8314636da89 Jul 15 16:59:13.258002 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:13.257547 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" event={"ID":"b41c58adddba08521de05a44de7576c8","Type":"ContainerStarted","Data":"8c056728bb455d3b12d7a12f3e1f624d71a7c931af5e735c991018c15f44976f"} Jul 15 16:59:13.259051 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:13.259018 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" event={"ID":"9ab826df-bc9b-42a8-bb4a-989adbec2594","Type":"ContainerStarted","Data":"8f66227489c9cde7b764ac25ee24a5042b50e68faf1c37ebc934f8314636da89"} Jul 15 16:59:13.260319 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:13.260288 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l46zz" event={"ID":"435df3eb-d81b-471e-b851-dbb91a35dbce","Type":"ContainerStarted","Data":"909928ee77a6cc6d3b6c013f545c1423a4b0f6ac8044bd318378d88dbb52101a"} Jul 15 16:59:13.261478 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:13.261457 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" event={"ID":"c5b4654a-99b4-4147-b754-4ce04a4a2ac0","Type":"ContainerStarted","Data":"2945347e85daa81a116b319a340dad72e86824cc493f79b3bbbf01360888d54e"} Jul 15 16:59:13.262769 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:13.262743 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" event={"ID":"9a0bd977-5444-4eb2-b422-f45950511b16","Type":"ContainerStarted","Data":"b19a684b148a27a41bf4ed6ca16b63fa2d95c6874f864b6365149caf08e012f5"} Jul 15 16:59:13.280240 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:13.280184 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-132.ec2.internal" podStartSLOduration=1.280164123 podStartE2EDuration="1.280164123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:13.280090507 +0000 UTC m=+3.522085739" watchObservedRunningTime="2026-07-15 16:59:13.280164123 +0000 UTC m=+3.522159354" Jul 15 16:59:15.270364 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:15.270234 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" event={"ID":"1176b310d9fb52e4fefed71966d7b67a","Type":"ContainerStarted","Data":"cfd4554e795f62cf082ec6354d4086fd9a6b315482e1294312d22b41d6e05b1f"} Jul 15 16:59:20.281075 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:20.280977 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l46zz" event={"ID":"435df3eb-d81b-471e-b851-dbb91a35dbce","Type":"ContainerStarted","Data":"3c1f1987dd242e9ec00d874fb65a97db0a7213939d6035275f9e1d6104b22f21"} Jul 15 16:59:20.283321 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:20.283284 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" event={"ID":"c5b4654a-99b4-4147-b754-4ce04a4a2ac0","Type":"ContainerStarted","Data":"cc062f4a2837af927e6b694e53cb39b34e239b5c69dfb3de16784e67752f9c84"} Jul 15 16:59:20.285031 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:20.285003 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" event={"ID":"9a0bd977-5444-4eb2-b422-f45950511b16","Type":"ContainerStarted","Data":"414087922f4ddb4052f8d323f34e274beb49be9b7943d8a66ca72b8bad9b7dbd"} Jul 15 16:59:20.305511 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:20.305459 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-l46zz" podStartSLOduration=2.294454349 podStartE2EDuration="9.305441239s" totalImagesPullingTime="7.01098689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:20.304922472 +0000 UTC m=+10.546917704" watchObservedRunningTime="2026-07-15 16:59:20.305441239 +0000 UTC m=+10.547436470" Jul 15 16:59:20.335227 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:20.335180 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vvw4b" podStartSLOduration=2.30476502 podStartE2EDuration="9.335161606s" totalImagesPullingTime="7.030396586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:20.333868542 +0000 UTC m=+10.575863772" watchObservedRunningTime="2026-07-15 16:59:20.335161606 +0000 UTC m=+10.577156835" Jul 15 16:59:21.222988 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:21.222938 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 16:59:21.292108 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:21.291985 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" event={"ID":"9a0bd977-5444-4eb2-b422-f45950511b16","Type":"ContainerStarted","Data":"acd7bb5038194d637db98093b7fbdeee44725d96e05ec8b6d3cfded081454312"} Jul 15 16:59:22.110344 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:22.110224 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T16:59:21.222960348Z","UUID":"2fada4a6-6f26-4b89-a5ec-52fe3381c517","Handler":null,"Name":"","Endpoint":""} Jul 15 16:59:22.112013 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:22.111991 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 16:59:22.112151 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:22.112044 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 16:59:26.301326 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:26.301129 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal_1176b310d9fb52e4fefed71966d7b67a/setup/0.log" Jul 15 16:59:26.301326 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:26.301302 2562 generic.go:356] "Generic (PLEG): container finished" podID="1176b310d9fb52e4fefed71966d7b67a" containerID="cfd4554e795f62cf082ec6354d4086fd9a6b315482e1294312d22b41d6e05b1f" exitCode=1 Jul 15 16:59:26.301699 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:26.301345 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" event={"ID":"1176b310d9fb52e4fefed71966d7b67a","Type":"ContainerDied","Data":"cfd4554e795f62cf082ec6354d4086fd9a6b315482e1294312d22b41d6e05b1f"} Jul 15 16:59:27.306765 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:27.306738 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal_1176b310d9fb52e4fefed71966d7b67a/setup/0.log" Jul 15 16:59:27.307710 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:27.306826 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" event={"ID":"1176b310d9fb52e4fefed71966d7b67a","Type":"ContainerStarted","Data":"ac7c5c90d03cfe895f05790407cecfb955a79207ce086f7d37fe2d95e435bfad"} Jul 15 16:59:27.309700 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:27.309667 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" event={"ID":"9ab826df-bc9b-42a8-bb4a-989adbec2594","Type":"ContainerStarted","Data":"e58475e5c59a67a26e070cfb87b3c04dc3da9c3fd285a185d26d843e710ebb17"} Jul 15 16:59:27.309820 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:27.309741 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" event={"ID":"9ab826df-bc9b-42a8-bb4a-989adbec2594","Type":"ContainerStarted","Data":"22309d1cf8139eea69bbee8bea2d0fa39a93aa0ffb5ee11b5ff78a22926a510b"} Jul 15 16:59:27.309820 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:27.309772 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" event={"ID":"9ab826df-bc9b-42a8-bb4a-989adbec2594","Type":"ContainerStarted","Data":"9b6a1af0da64aaae010a53da0d808c5c5524c0386952a56b11815f64595f022a"} Jul 15 16:59:27.309820 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:27.309807 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" event={"ID":"9ab826df-bc9b-42a8-bb4a-989adbec2594","Type":"ContainerStarted","Data":"299fcf93c894c7ba7aeb56fc89533731c0e953f910e115962e81e634ebb23e4e"} Jul 15 16:59:27.309955 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:27.309852 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" event={"ID":"9ab826df-bc9b-42a8-bb4a-989adbec2594","Type":"ContainerStarted","Data":"9b9c8ad44fcfb0fca6a1e93e95924fd91ec94ba3c8ed9fca1c5ffaaaa5c7a150"} Jul 15 16:59:27.309955 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:27.309920 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" event={"ID":"9ab826df-bc9b-42a8-bb4a-989adbec2594","Type":"ContainerStarted","Data":"fd60d48374897f0575bf8ff290af2d3bf5070efcfc7c0f1c05db1d7a8d69c940"} Jul 15 16:59:27.311583 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:27.311560 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" event={"ID":"9a0bd977-5444-4eb2-b422-f45950511b16","Type":"ContainerStarted","Data":"1efe20c128dc4b797b76779d00b6a78c21490b8a7d273f1e66ff822cdd293d60"} Jul 15 16:59:27.352581 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:27.352545 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqnlr" podStartSLOduration=3.219255488 podStartE2EDuration="16.352533199s" totalImagesPullingTime="13.133277711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:27.352483502 +0000 UTC m=+17.594478727" watchObservedRunningTime="2026-07-15 16:59:27.352533199 +0000 UTC m=+17.594528415" Jul 15 16:59:29.317234 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:29.317043 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" event={"ID":"9ab826df-bc9b-42a8-bb4a-989adbec2594","Type":"ContainerStarted","Data":"447ddb8c57ed9319597d5d6105224533ed28b506e73fe109cbedd0626f3130fd"} Jul 15 16:59:31.323995 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:31.323958 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" event={"ID":"9ab826df-bc9b-42a8-bb4a-989adbec2594","Type":"ContainerStarted","Data":"b2e2c4f31f52832ca2bcbf70e032a4ffa7a0f6396a5251a9d2ad64cba7c015a4"} Jul 15 16:59:31.324475 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:31.324241 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:31.324475 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:31.324307 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:31.339895 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:31.339875 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:31.378258 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:31.378213 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" podStartSLOduration=7.062915683 podStartE2EDuration="20.378181567s" totalImagesPullingTime="13.315265884s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:31.378004772 +0000 UTC m=+21.619999998" watchObservedRunningTime="2026-07-15 16:59:31.378181567 +0000 UTC m=+21.620176785" Jul 15 16:59:32.326099 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:32.326061 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:32.340650 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:32.340619 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 16:59:36.802106 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:36.802077 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sd27v" Jul 15 16:59:36.977718 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:36.977671 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-zgqgp"] Jul 15 16:59:36.984534 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:36.984502 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zgqgp" Jul 15 16:59:36.988939 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:36.988898 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 16:59:36.988939 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:36.988902 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-84ncz\"" Jul 15 16:59:36.989336 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:36.989315 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:59:36.991259 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:36.991235 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 16:59:37.015092 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.015057 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6r8xd"] Jul 15 16:59:37.018054 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.018037 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:37.018159 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.018108 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6r8xd" podUID="f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca" Jul 15 16:59:37.078529 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.078504 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-cxs9l"] Jul 15 16:59:37.082024 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.081559 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.084063 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.083626 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-sqd2w"] Jul 15 16:59:37.086055 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.085966 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 16:59:37.086178 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.086115 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pjc4n\"" Jul 15 16:59:37.086178 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.086139 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 16:59:37.086178 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.086167 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 16:59:37.086348 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.086180 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 16:59:37.086348 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.086190 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 16:59:37.086886 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.086681 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sqd2w" Jul 15 16:59:37.092501 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.092472 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 16:59:37.092736 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.092698 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 16:59:37.093772 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.093746 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-42kjl\"" Jul 15 16:59:37.120331 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120301 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.120477 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120333 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhbm6\" (UniqueName: \"kubernetes.io/projected/35c01559-3824-4e5c-bfea-71ead02ea8ed-kube-api-access-hhbm6\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.120477 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120355 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwwfl\" (UniqueName: \"kubernetes.io/projected/e5bf8ca0-c632-4f9c-a403-9f16133b4651-kube-api-access-fwwfl\") pod \"iptables-alerter-zgqgp\" (UID: \"e5bf8ca0-c632-4f9c-a403-9f16133b4651\") " pod="openshift-network-operator/iptables-alerter-zgqgp" Jul 15 16:59:37.120477 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120372 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-kubelet-config\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:37.120477 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120388 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-system-cni-dir\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.120477 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120432 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-dbus\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:37.120477 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120459 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:37.120477 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120476 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-os-release\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.120748 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/35c01559-3824-4e5c-bfea-71ead02ea8ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.120748 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120584 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5bf8ca0-c632-4f9c-a403-9f16133b4651-iptables-alerter-script\") pod \"iptables-alerter-zgqgp\" (UID: \"e5bf8ca0-c632-4f9c-a403-9f16133b4651\") " pod="openshift-network-operator/iptables-alerter-zgqgp" Jul 15 16:59:37.120748 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120610 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35c01559-3824-4e5c-bfea-71ead02ea8ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.120748 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5bf8ca0-c632-4f9c-a403-9f16133b4651-host-slash\") pod \"iptables-alerter-zgqgp\" (UID: \"e5bf8ca0-c632-4f9c-a403-9f16133b4651\") " pod="openshift-network-operator/iptables-alerter-zgqgp" Jul 15 16:59:37.120748 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120665 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-cnibin\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.120748 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.120681 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/35c01559-3824-4e5c-bfea-71ead02ea8ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.132473 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.132445 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-r92qp"] Jul 15 16:59:37.135415 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.135398 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:37.135506 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.135471 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r92qp" podUID="07721dc7-4474-4c7f-b037-878a06841f31" Jul 15 16:59:37.160390 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.160367 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-87dcf"] Jul 15 16:59:37.163061 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.163026 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-87dcf" Jul 15 16:59:37.169118 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.169101 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t9hrp\"" Jul 15 16:59:37.169351 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.169326 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 16:59:37.170133 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.170117 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 16:59:37.176293 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.176276 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-w5h4s"] Jul 15 16:59:37.178963 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.178950 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.184252 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.184191 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 16:59:37.188387 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.188369 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6njq9\"" Jul 15 16:59:37.197090 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.197073 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-qwxnw"] Jul 15 16:59:37.199804 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.199786 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:37.199882 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.199864 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qwxnw" podUID="8dd93cd1-bc16-44a6-83fb-a0861aa19e26" Jul 15 16:59:37.221794 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.221774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/35c01559-3824-4e5c-bfea-71ead02ea8ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.221900 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.221805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5bf8ca0-c632-4f9c-a403-9f16133b4651-iptables-alerter-script\") pod \"iptables-alerter-zgqgp\" (UID: \"e5bf8ca0-c632-4f9c-a403-9f16133b4651\") " pod="openshift-network-operator/iptables-alerter-zgqgp" Jul 15 16:59:37.221900 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.221824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35c01559-3824-4e5c-bfea-71ead02ea8ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.221900 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.221864 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-daemon-config\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.221900 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.221897 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5bf8ca0-c632-4f9c-a403-9f16133b4651-host-slash\") pod \"iptables-alerter-zgqgp\" (UID: \"e5bf8ca0-c632-4f9c-a403-9f16133b4651\") " pod="openshift-network-operator/iptables-alerter-zgqgp" Jul 15 16:59:37.222100 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.221921 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f19cf2f1-fb21-4d7f-add5-939f1dec2fca-agent-certs\") pod \"konnectivity-agent-87dcf\" (UID: \"f19cf2f1-fb21-4d7f-add5-939f1dec2fca\") " pod="kube-system/konnectivity-agent-87dcf" Jul 15 16:59:37.222100 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.221943 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-system-cni-dir\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.222100 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.221964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-run-netns\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.222100 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.221983 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-var-lib-cni-bin\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.222100 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222010 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5bf8ca0-c632-4f9c-a403-9f16133b4651-host-slash\") pod \"iptables-alerter-zgqgp\" (UID: \"e5bf8ca0-c632-4f9c-a403-9f16133b4651\") " pod="openshift-network-operator/iptables-alerter-zgqgp" Jul 15 16:59:37.222100 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222038 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-cnibin\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.222100 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222072 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/35c01559-3824-4e5c-bfea-71ead02ea8ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.222100 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a700f8f-0ac0-4af0-a190-c8865d5db3e9-tmp-dir\") pod \"node-resolver-sqd2w\" (UID: \"2a700f8f-0ac0-4af0-a190-c8865d5db3e9\") " pod="openshift-dns/node-resolver-sqd2w" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-cnibin\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222126 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-socket-dir-parent\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222171 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-cni-dir\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222199 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rgsw\" (UniqueName: \"kubernetes.io/projected/652c50d6-b43e-4f5c-8cc2-70d2155a9188-kube-api-access-4rgsw\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222264 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv2nv\" (UniqueName: \"kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv\") pod \"network-check-target-r92qp\" (UID: \"07721dc7-4474-4c7f-b037-878a06841f31\") " pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222292 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhbm6\" (UniqueName: \"kubernetes.io/projected/35c01559-3824-4e5c-bfea-71ead02ea8ed-kube-api-access-hhbm6\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222312 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m766f\" (UniqueName: \"kubernetes.io/projected/2a700f8f-0ac0-4af0-a190-c8865d5db3e9-kube-api-access-m766f\") pod \"node-resolver-sqd2w\" (UID: \"2a700f8f-0ac0-4af0-a190-c8865d5db3e9\") " pod="openshift-dns/node-resolver-sqd2w" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222333 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-conf-dir\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222362 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwwfl\" (UniqueName: \"kubernetes.io/projected/e5bf8ca0-c632-4f9c-a403-9f16133b4651-kube-api-access-fwwfl\") pod \"iptables-alerter-zgqgp\" (UID: \"e5bf8ca0-c632-4f9c-a403-9f16133b4651\") " pod="openshift-network-operator/iptables-alerter-zgqgp" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222388 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f19cf2f1-fb21-4d7f-add5-939f1dec2fca-konnectivity-ca\") pod \"konnectivity-agent-87dcf\" (UID: \"f19cf2f1-fb21-4d7f-add5-939f1dec2fca\") " pod="kube-system/konnectivity-agent-87dcf" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222410 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-os-release\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-kubelet-config\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:37.222457 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222460 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-system-cni-dir\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222468 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/35c01559-3824-4e5c-bfea-71ead02ea8ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-var-lib-kubelet\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222460 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5bf8ca0-c632-4f9c-a403-9f16133b4651-iptables-alerter-script\") pod \"iptables-alerter-zgqgp\" (UID: \"e5bf8ca0-c632-4f9c-a403-9f16133b4651\") " pod="openshift-network-operator/iptables-alerter-zgqgp" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222545 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-kubelet-config\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222587 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-hostroot\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-run-multus-certs\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222619 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35c01559-3824-4e5c-bfea-71ead02ea8ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222637 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222665 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-system-cni-dir\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-dbus\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222732 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222743 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/35c01559-3824-4e5c-bfea-71ead02ea8ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222784 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/652c50d6-b43e-4f5c-8cc2-70d2155a9188-cni-binary-copy\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.222804 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222823 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-os-release\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.223047 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222873 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a700f8f-0ac0-4af0-a190-c8865d5db3e9-hosts-file\") pod \"node-resolver-sqd2w\" (UID: \"2a700f8f-0ac0-4af0-a190-c8865d5db3e9\") " pod="openshift-dns/node-resolver-sqd2w" Jul 15 16:59:37.223486 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.222892 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret podName:f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca nodeName:}" failed. No retries permitted until 2026-07-15 16:59:37.722872718 +0000 UTC m=+27.964867937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret") pod "global-pull-secret-syncer-6r8xd" (UID: "f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:59:37.223486 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222892 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-dbus\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:37.223486 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222894 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35c01559-3824-4e5c-bfea-71ead02ea8ed-os-release\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.223486 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-etc-kubernetes\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.223486 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.222998 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-cnibin\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.223486 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.223022 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-run-k8s-cni-cncf-io\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.223486 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.223046 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-var-lib-cni-multus\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.239507 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.239483 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwwfl\" (UniqueName: \"kubernetes.io/projected/e5bf8ca0-c632-4f9c-a403-9f16133b4651-kube-api-access-fwwfl\") pod \"iptables-alerter-zgqgp\" (UID: \"e5bf8ca0-c632-4f9c-a403-9f16133b4651\") " pod="openshift-network-operator/iptables-alerter-zgqgp" Jul 15 16:59:37.241089 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.241073 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhbm6\" (UniqueName: \"kubernetes.io/projected/35c01559-3824-4e5c-bfea-71ead02ea8ed-kube-api-access-hhbm6\") pod \"multus-additional-cni-plugins-cxs9l\" (UID: \"35c01559-3824-4e5c-bfea-71ead02ea8ed\") " pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.295558 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.295532 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zgqgp" Jul 15 16:59:37.306564 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:37.306529 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5bf8ca0_c632_4f9c_a403_9f16133b4651.slice/crio-38b0ba04f6afa2a7d29b236ba8a061bee31ff21d98d608b5e04219141ce1b9d2 WatchSource:0}: Error finding container 38b0ba04f6afa2a7d29b236ba8a061bee31ff21d98d608b5e04219141ce1b9d2: Status 404 returned error can't find the container with id 38b0ba04f6afa2a7d29b236ba8a061bee31ff21d98d608b5e04219141ce1b9d2 Jul 15 16:59:37.323943 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.323917 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a700f8f-0ac0-4af0-a190-c8865d5db3e9-hosts-file\") pod \"node-resolver-sqd2w\" (UID: \"2a700f8f-0ac0-4af0-a190-c8865d5db3e9\") " pod="openshift-dns/node-resolver-sqd2w" Jul 15 16:59:37.324028 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.323949 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-etc-kubernetes\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324028 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.323975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-cnibin\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324120 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324034 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a700f8f-0ac0-4af0-a190-c8865d5db3e9-hosts-file\") pod \"node-resolver-sqd2w\" (UID: \"2a700f8f-0ac0-4af0-a190-c8865d5db3e9\") " pod="openshift-dns/node-resolver-sqd2w" Jul 15 16:59:37.324120 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324042 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-cnibin\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324120 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324035 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-etc-kubernetes\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324120 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324056 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-run-k8s-cni-cncf-io\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324120 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324076 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-run-k8s-cni-cncf-io\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324120 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-var-lib-cni-multus\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324120 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324108 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-var-lib-cni-multus\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324120 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324112 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6g8l\" (UniqueName: \"kubernetes.io/projected/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-kube-api-access-x6g8l\") pod \"network-metrics-daemon-qwxnw\" (UID: \"8dd93cd1-bc16-44a6-83fb-a0861aa19e26\") " pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324137 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-daemon-config\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f19cf2f1-fb21-4d7f-add5-939f1dec2fca-agent-certs\") pod \"konnectivity-agent-87dcf\" (UID: \"f19cf2f1-fb21-4d7f-add5-939f1dec2fca\") " pod="kube-system/konnectivity-agent-87dcf" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-system-cni-dir\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-run-netns\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-var-lib-cni-bin\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a700f8f-0ac0-4af0-a190-c8865d5db3e9-tmp-dir\") pod \"node-resolver-sqd2w\" (UID: \"2a700f8f-0ac0-4af0-a190-c8865d5db3e9\") " pod="openshift-dns/node-resolver-sqd2w" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324264 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-system-cni-dir\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324263 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-socket-dir-parent\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324298 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-run-netns\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324330 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-var-lib-cni-bin\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-cni-dir\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324367 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-socket-dir-parent\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rgsw\" (UniqueName: \"kubernetes.io/projected/652c50d6-b43e-4f5c-8cc2-70d2155a9188-kube-api-access-4rgsw\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv2nv\" (UniqueName: \"kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv\") pod \"network-check-target-r92qp\" (UID: \"07721dc7-4474-4c7f-b037-878a06841f31\") " pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:37.324440 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324434 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-cni-dir\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m766f\" (UniqueName: \"kubernetes.io/projected/2a700f8f-0ac0-4af0-a190-c8865d5db3e9-kube-api-access-m766f\") pod \"node-resolver-sqd2w\" (UID: \"2a700f8f-0ac0-4af0-a190-c8865d5db3e9\") " pod="openshift-dns/node-resolver-sqd2w" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324474 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-conf-dir\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f19cf2f1-fb21-4d7f-add5-939f1dec2fca-konnectivity-ca\") pod \"konnectivity-agent-87dcf\" (UID: \"f19cf2f1-fb21-4d7f-add5-939f1dec2fca\") " pod="kube-system/konnectivity-agent-87dcf" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-os-release\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324558 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-var-lib-kubelet\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324606 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a700f8f-0ac0-4af0-a190-c8865d5db3e9-tmp-dir\") pod \"node-resolver-sqd2w\" (UID: \"2a700f8f-0ac0-4af0-a190-c8865d5db3e9\") " pod="openshift-dns/node-resolver-sqd2w" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324594 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs\") pod \"network-metrics-daemon-qwxnw\" (UID: \"8dd93cd1-bc16-44a6-83fb-a0861aa19e26\") " pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324632 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-conf-dir\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-hostroot\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-run-multus-certs\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324688 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-hostroot\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324731 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/652c50d6-b43e-4f5c-8cc2-70d2155a9188-cni-binary-copy\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324736 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-var-lib-kubelet\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324760 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-os-release\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324798 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/652c50d6-b43e-4f5c-8cc2-70d2155a9188-host-run-multus-certs\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325109 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.324867 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/652c50d6-b43e-4f5c-8cc2-70d2155a9188-multus-daemon-config\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.325651 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.325135 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f19cf2f1-fb21-4d7f-add5-939f1dec2fca-konnectivity-ca\") pod \"konnectivity-agent-87dcf\" (UID: \"f19cf2f1-fb21-4d7f-add5-939f1dec2fca\") " pod="kube-system/konnectivity-agent-87dcf" Jul 15 16:59:37.325651 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.325275 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/652c50d6-b43e-4f5c-8cc2-70d2155a9188-cni-binary-copy\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.327014 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.326996 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f19cf2f1-fb21-4d7f-add5-939f1dec2fca-agent-certs\") pod \"konnectivity-agent-87dcf\" (UID: \"f19cf2f1-fb21-4d7f-add5-939f1dec2fca\") " pod="kube-system/konnectivity-agent-87dcf" Jul 15 16:59:37.334051 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.333990 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zgqgp" event={"ID":"e5bf8ca0-c632-4f9c-a403-9f16133b4651","Type":"ContainerStarted","Data":"38b0ba04f6afa2a7d29b236ba8a061bee31ff21d98d608b5e04219141ce1b9d2"} Jul 15 16:59:37.343183 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.343161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rgsw\" (UniqueName: \"kubernetes.io/projected/652c50d6-b43e-4f5c-8cc2-70d2155a9188-kube-api-access-4rgsw\") pod \"multus-w5h4s\" (UID: \"652c50d6-b43e-4f5c-8cc2-70d2155a9188\") " pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.343531 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.343515 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:59:37.343576 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.343533 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:59:37.343576 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.343543 2562 projected.go:196] Error preparing data for projected volume kube-api-access-lv2nv for pod openshift-network-diagnostics/network-check-target-r92qp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:59:37.343643 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.343588 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv podName:07721dc7-4474-4c7f-b037-878a06841f31 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:37.843576204 +0000 UTC m=+28.085571407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lv2nv" (UniqueName: "kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv") pod "network-check-target-r92qp" (UID: "07721dc7-4474-4c7f-b037-878a06841f31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:59:37.344034 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.344015 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m766f\" (UniqueName: \"kubernetes.io/projected/2a700f8f-0ac0-4af0-a190-c8865d5db3e9-kube-api-access-m766f\") pod \"node-resolver-sqd2w\" (UID: \"2a700f8f-0ac0-4af0-a190-c8865d5db3e9\") " pod="openshift-dns/node-resolver-sqd2w" Jul 15 16:59:37.394736 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.394713 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cxs9l" Jul 15 16:59:37.403202 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.403184 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sqd2w" Jul 15 16:59:37.405526 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:37.405505 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c01559_3824_4e5c_bfea_71ead02ea8ed.slice/crio-a754c4ebfa414295c2374f48606647db535d7cdaee91a05c6e19964c97b83a55 WatchSource:0}: Error finding container a754c4ebfa414295c2374f48606647db535d7cdaee91a05c6e19964c97b83a55: Status 404 returned error can't find the container with id a754c4ebfa414295c2374f48606647db535d7cdaee91a05c6e19964c97b83a55 Jul 15 16:59:37.411510 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:37.411492 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a700f8f_0ac0_4af0_a190_c8865d5db3e9.slice/crio-d392ced6e587c4e6f3ca0a9b8921696cf28e7f01e9f4e6535e076087e2733b20 WatchSource:0}: Error finding container d392ced6e587c4e6f3ca0a9b8921696cf28e7f01e9f4e6535e076087e2733b20: Status 404 returned error can't find the container with id d392ced6e587c4e6f3ca0a9b8921696cf28e7f01e9f4e6535e076087e2733b20 Jul 15 16:59:37.425126 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.425108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6g8l\" (UniqueName: \"kubernetes.io/projected/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-kube-api-access-x6g8l\") pod \"network-metrics-daemon-qwxnw\" (UID: \"8dd93cd1-bc16-44a6-83fb-a0861aa19e26\") " pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:37.425184 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.425163 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs\") pod \"network-metrics-daemon-qwxnw\" (UID: \"8dd93cd1-bc16-44a6-83fb-a0861aa19e26\") " pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:37.425278 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.425263 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:59:37.425337 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.425325 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs podName:8dd93cd1-bc16-44a6-83fb-a0861aa19e26 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:37.925307081 +0000 UTC m=+28.167302302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs") pod "network-metrics-daemon-qwxnw" (UID: "8dd93cd1-bc16-44a6-83fb-a0861aa19e26") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:59:37.438446 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.438427 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6g8l\" (UniqueName: \"kubernetes.io/projected/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-kube-api-access-x6g8l\") pod \"network-metrics-daemon-qwxnw\" (UID: \"8dd93cd1-bc16-44a6-83fb-a0861aa19e26\") " pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:37.474141 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.474112 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-87dcf" Jul 15 16:59:37.486753 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.486724 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w5h4s" Jul 15 16:59:37.491578 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:37.491559 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19cf2f1_fb21_4d7f_add5_939f1dec2fca.slice/crio-2280bd31dfb9bea5389afe5cdc988c623bc6d883f68a4ebd39d2b43763c7f644 WatchSource:0}: Error finding container 2280bd31dfb9bea5389afe5cdc988c623bc6d883f68a4ebd39d2b43763c7f644: Status 404 returned error can't find the container with id 2280bd31dfb9bea5389afe5cdc988c623bc6d883f68a4ebd39d2b43763c7f644 Jul 15 16:59:37.495241 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:37.495221 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652c50d6_b43e_4f5c_8cc2_70d2155a9188.slice/crio-8bba9f4be1d3b2b806e4bcf27811c27001f67561e67af83aaa0efc8bf1b13dc4 WatchSource:0}: Error finding container 8bba9f4be1d3b2b806e4bcf27811c27001f67561e67af83aaa0efc8bf1b13dc4: Status 404 returned error can't find the container with id 8bba9f4be1d3b2b806e4bcf27811c27001f67561e67af83aaa0efc8bf1b13dc4 Jul 15 16:59:37.728404 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.728325 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:37.728536 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.728461 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:59:37.728536 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.728513 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret podName:f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca nodeName:}" failed. No retries permitted until 2026-07-15 16:59:38.728500072 +0000 UTC m=+28.970495276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret") pod "global-pull-secret-syncer-6r8xd" (UID: "f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:59:37.807560 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.807512 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:54:36 +0000 UTC" deadline="2028-02-25 14:41:56.208328189 +0000 UTC" Jul 15 16:59:37.807560 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.807554 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14157h42m18.400777744s" Jul 15 16:59:37.929952 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.929829 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv2nv\" (UniqueName: \"kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv\") pod \"network-check-target-r92qp\" (UID: \"07721dc7-4474-4c7f-b037-878a06841f31\") " pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:37.929952 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:37.929912 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs\") pod \"network-metrics-daemon-qwxnw\" (UID: \"8dd93cd1-bc16-44a6-83fb-a0861aa19e26\") " pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:37.930153 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.930076 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:59:37.930153 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.930098 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:59:37.930153 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.930110 2562 projected.go:196] Error preparing data for projected volume kube-api-access-lv2nv for pod openshift-network-diagnostics/network-check-target-r92qp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:59:37.930153 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.930129 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:59:37.930340 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.930175 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv podName:07721dc7-4474-4c7f-b037-878a06841f31 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:38.930153654 +0000 UTC m=+29.172148870 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lv2nv" (UniqueName: "kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv") pod "network-check-target-r92qp" (UID: "07721dc7-4474-4c7f-b037-878a06841f31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:59:37.930340 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:37.930196 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs podName:8dd93cd1-bc16-44a6-83fb-a0861aa19e26 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:38.930185663 +0000 UTC m=+29.172180876 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs") pod "network-metrics-daemon-qwxnw" (UID: "8dd93cd1-bc16-44a6-83fb-a0861aa19e26") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:59:38.338196 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.338163 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w5h4s" event={"ID":"652c50d6-b43e-4f5c-8cc2-70d2155a9188","Type":"ContainerStarted","Data":"8bba9f4be1d3b2b806e4bcf27811c27001f67561e67af83aaa0efc8bf1b13dc4"} Jul 15 16:59:38.342024 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.341993 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-87dcf" event={"ID":"f19cf2f1-fb21-4d7f-add5-939f1dec2fca","Type":"ContainerStarted","Data":"2280bd31dfb9bea5389afe5cdc988c623bc6d883f68a4ebd39d2b43763c7f644"} Jul 15 16:59:38.346006 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.345979 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxs9l" event={"ID":"35c01559-3824-4e5c-bfea-71ead02ea8ed","Type":"ContainerStarted","Data":"a754c4ebfa414295c2374f48606647db535d7cdaee91a05c6e19964c97b83a55"} Jul 15 16:59:38.348119 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.348097 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal_1176b310d9fb52e4fefed71966d7b67a/setup/1.log" Jul 15 16:59:38.348524 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.348503 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal_1176b310d9fb52e4fefed71966d7b67a/setup/0.log" Jul 15 16:59:38.348636 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.348542 2562 generic.go:356] "Generic (PLEG): container finished" podID="1176b310d9fb52e4fefed71966d7b67a" containerID="ac7c5c90d03cfe895f05790407cecfb955a79207ce086f7d37fe2d95e435bfad" exitCode=1 Jul 15 16:59:38.348636 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.348613 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" event={"ID":"1176b310d9fb52e4fefed71966d7b67a","Type":"ContainerDied","Data":"ac7c5c90d03cfe895f05790407cecfb955a79207ce086f7d37fe2d95e435bfad"} Jul 15 16:59:38.348747 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.348654 2562 scope.go:117] "RemoveContainer" containerID="cfd4554e795f62cf082ec6354d4086fd9a6b315482e1294312d22b41d6e05b1f" Jul 15 16:59:38.349113 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.349096 2562 scope.go:117] "RemoveContainer" containerID="cfd4554e795f62cf082ec6354d4086fd9a6b315482e1294312d22b41d6e05b1f" Jul 15 16:59:38.356197 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.356173 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sqd2w" event={"ID":"2a700f8f-0ac0-4af0-a190-c8865d5db3e9","Type":"ContainerStarted","Data":"d392ced6e587c4e6f3ca0a9b8921696cf28e7f01e9f4e6535e076087e2733b20"} Jul 15 16:59:38.380354 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.380316 2562 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal_openshift-machine-config-operator_1176b310d9fb52e4fefed71966d7b67a_0 in pod sandbox aefd244820935d0aaa9df301738ada05b7405d59c740e247050e63a6abe2ce3a from index: no such id: 'cfd4554e795f62cf082ec6354d4086fd9a6b315482e1294312d22b41d6e05b1f'" containerID="cfd4554e795f62cf082ec6354d4086fd9a6b315482e1294312d22b41d6e05b1f" Jul 15 16:59:38.380450 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.380379 2562 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal_openshift-machine-config-operator_1176b310d9fb52e4fefed71966d7b67a_0 in pod sandbox aefd244820935d0aaa9df301738ada05b7405d59c740e247050e63a6abe2ce3a from index: no such id: 'cfd4554e795f62cf082ec6354d4086fd9a6b315482e1294312d22b41d6e05b1f'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal_openshift-machine-config-operator(1176b310d9fb52e4fefed71966d7b67a)\"" logger="UnhandledError" Jul 15 16:59:38.380659 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.380634 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal_openshift-machine-config-operator(1176b310d9fb52e4fefed71966d7b67a)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" podUID="1176b310d9fb52e4fefed71966d7b67a" Jul 15 16:59:38.484338 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.484252 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6r8xd"] Jul 15 16:59:38.484485 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.484388 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:38.484538 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.484489 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6r8xd" podUID="f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca" Jul 15 16:59:38.496720 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.494899 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qwxnw"] Jul 15 16:59:38.496720 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.495117 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:38.496720 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.495291 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qwxnw" podUID="8dd93cd1-bc16-44a6-83fb-a0861aa19e26" Jul 15 16:59:38.498198 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.498171 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r92qp"] Jul 15 16:59:38.498316 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.498275 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:38.498375 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.498357 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r92qp" podUID="07721dc7-4474-4c7f-b037-878a06841f31" Jul 15 16:59:38.750214 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.750142 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:38.750352 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.750315 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:59:38.750416 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.750393 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret podName:f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca nodeName:}" failed. No retries permitted until 2026-07-15 16:59:40.750371202 +0000 UTC m=+30.992366424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret") pod "global-pull-secret-syncer-6r8xd" (UID: "f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:59:38.807859 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.807777 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:54:36 +0000 UTC" deadline="2027-12-30 01:43:54.486782773 +0000 UTC" Jul 15 16:59:38.807859 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.807853 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12776h44m15.678935114s" Jul 15 16:59:38.951325 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.951299 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv2nv\" (UniqueName: \"kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv\") pod \"network-check-target-r92qp\" (UID: \"07721dc7-4474-4c7f-b037-878a06841f31\") " pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:38.951459 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:38.951349 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs\") pod \"network-metrics-daemon-qwxnw\" (UID: \"8dd93cd1-bc16-44a6-83fb-a0861aa19e26\") " pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:38.951510 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.951476 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:59:38.951561 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.951525 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs podName:8dd93cd1-bc16-44a6-83fb-a0861aa19e26 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:40.951510516 +0000 UTC m=+31.193505719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs") pod "network-metrics-daemon-qwxnw" (UID: "8dd93cd1-bc16-44a6-83fb-a0861aa19e26") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:59:38.951910 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.951764 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:59:38.951910 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.951792 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:59:38.951910 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.951805 2562 projected.go:196] Error preparing data for projected volume kube-api-access-lv2nv for pod openshift-network-diagnostics/network-check-target-r92qp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:59:38.951910 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:38.951886 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv podName:07721dc7-4474-4c7f-b037-878a06841f31 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:40.951867889 +0000 UTC m=+31.193863112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lv2nv" (UniqueName: "kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv") pod "network-check-target-r92qp" (UID: "07721dc7-4474-4c7f-b037-878a06841f31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:59:39.361034 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:39.361000 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal_1176b310d9fb52e4fefed71966d7b67a/setup/1.log" Jul 15 16:59:40.248757 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:40.248067 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:40.248757 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:40.248181 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6r8xd" podUID="f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca" Jul 15 16:59:40.248757 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:40.248205 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:40.248757 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:40.248317 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r92qp" podUID="07721dc7-4474-4c7f-b037-878a06841f31" Jul 15 16:59:40.248757 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:40.248370 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:40.248757 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:40.248443 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qwxnw" podUID="8dd93cd1-bc16-44a6-83fb-a0861aa19e26" Jul 15 16:59:40.766541 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:40.766501 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:40.766715 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:40.766652 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:59:40.766791 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:40.766717 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret podName:f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca nodeName:}" failed. No retries permitted until 2026-07-15 16:59:44.766701288 +0000 UTC m=+35.008696495 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret") pod "global-pull-secret-syncer-6r8xd" (UID: "f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:59:40.968400 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:40.968209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv2nv\" (UniqueName: \"kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv\") pod \"network-check-target-r92qp\" (UID: \"07721dc7-4474-4c7f-b037-878a06841f31\") " pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:40.968400 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:40.968272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs\") pod \"network-metrics-daemon-qwxnw\" (UID: \"8dd93cd1-bc16-44a6-83fb-a0861aa19e26\") " pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:40.968400 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:40.968342 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:59:40.968400 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:40.968370 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:59:40.968400 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:40.968386 2562 projected.go:196] Error preparing data for projected volume kube-api-access-lv2nv for pod openshift-network-diagnostics/network-check-target-r92qp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:59:40.968746 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:40.968437 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv podName:07721dc7-4474-4c7f-b037-878a06841f31 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:44.968422958 +0000 UTC m=+35.210418163 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lv2nv" (UniqueName: "kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv") pod "network-check-target-r92qp" (UID: "07721dc7-4474-4c7f-b037-878a06841f31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:59:40.968991 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:40.968967 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:59:40.969105 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:40.969033 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs podName:8dd93cd1-bc16-44a6-83fb-a0861aa19e26 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:44.969016842 +0000 UTC m=+35.211012053 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs") pod "network-metrics-daemon-qwxnw" (UID: "8dd93cd1-bc16-44a6-83fb-a0861aa19e26") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:59:41.366971 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:41.366871 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sqd2w" event={"ID":"2a700f8f-0ac0-4af0-a190-c8865d5db3e9","Type":"ContainerStarted","Data":"1e2649a783be4be6408793b2be7cd8ba5f711d1149287b135fe3d6eb4ba7a208"} Jul 15 16:59:41.368520 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:41.368487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-87dcf" event={"ID":"f19cf2f1-fb21-4d7f-add5-939f1dec2fca","Type":"ContainerStarted","Data":"61172f74dbc4c94839ecc75fe1a8b037b8cb310502f5eced6a718d1cf891c350"} Jul 15 16:59:41.370080 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:41.370054 2562 generic.go:356] "Generic (PLEG): container finished" podID="35c01559-3824-4e5c-bfea-71ead02ea8ed" containerID="6bf0385aaac85b50144bc9cef2f1b7493eff1324f14f342684bb6d9e9ec7e70f" exitCode=0 Jul 15 16:59:41.370202 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:41.370104 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxs9l" event={"ID":"35c01559-3824-4e5c-bfea-71ead02ea8ed","Type":"ContainerDied","Data":"6bf0385aaac85b50144bc9cef2f1b7493eff1324f14f342684bb6d9e9ec7e70f"} Jul 15 16:59:41.438316 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:41.438085 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sqd2w" podStartSLOduration=0.834035851 podStartE2EDuration="4.438063128s" totalImagesPullingTime="3.604027277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:41.396736386 +0000 UTC m=+31.638731617" watchObservedRunningTime="2026-07-15 16:59:41.438063128 +0000 UTC m=+31.680058335" Jul 15 16:59:41.701854 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:41.701749 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:42.005583 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:42.005467 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:42.246370 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:42.245912 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:42.246370 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:42.245923 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:42.246370 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:42.246048 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6r8xd" podUID="f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca" Jul 15 16:59:42.246370 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:42.245923 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:42.246370 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:42.246157 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qwxnw" podUID="8dd93cd1-bc16-44a6-83fb-a0861aa19e26" Jul 15 16:59:42.246370 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:42.246241 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r92qp" podUID="07721dc7-4474-4c7f-b037-878a06841f31" Jul 15 16:59:42.302365 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:42.302275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:42.474737 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:42.474707 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-87dcf" Jul 15 16:59:42.475235 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:42.475201 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-87dcf" Jul 15 16:59:42.486128 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:42.486098 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:42.504641 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:42.504592 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-87dcf" podStartSLOduration=1.987518256 podStartE2EDuration="5.504572927s" totalImagesPullingTime="3.517054671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:41.480178431 +0000 UTC m=+31.722173659" watchObservedRunningTime="2026-07-15 16:59:42.504572927 +0000 UTC m=+32.746568150" Jul 15 16:59:42.766316 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:42.766285 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:42.989472 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:42.989188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:43.075243 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:43.075207 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:43.345534 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:43.345499 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:43.377270 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:43.377054 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zgqgp" event={"ID":"e5bf8ca0-c632-4f9c-a403-9f16133b4651","Type":"ContainerStarted","Data":"c008c60c7f63f100750b3378edae43dd3c48eb36da483a218b8919f300526de8"} Jul 15 16:59:43.401509 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:43.401434 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zgqgp" podStartSLOduration=3.700200477 podStartE2EDuration="7.401418621s" totalImagesPullingTime="3.701218144s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:43.401296831 +0000 UTC m=+33.643292058" watchObservedRunningTime="2026-07-15 16:59:43.401418621 +0000 UTC m=+33.643413847" Jul 15 16:59:43.485710 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:43.485665 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:43.648701 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:43.648622 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:43.748605 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:43.748562 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:44.245691 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:44.245624 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:44.245894 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:44.245631 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:44.245894 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:44.245760 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r92qp" podUID="07721dc7-4474-4c7f-b037-878a06841f31" Jul 15 16:59:44.245894 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:44.245796 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:44.246059 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:44.245925 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qwxnw" podUID="8dd93cd1-bc16-44a6-83fb-a0861aa19e26" Jul 15 16:59:44.246059 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:44.246008 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6r8xd" podUID="f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca" Jul 15 16:59:44.379676 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:44.379646 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 16:59:44.530707 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:44.530624 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:44.734953 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:44.734918 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:44.800990 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:44.800915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:44.801150 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:44.801037 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:59:44.801150 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:44.801107 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret podName:f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca nodeName:}" failed. No retries permitted until 2026-07-15 16:59:52.801083934 +0000 UTC m=+43.043079141 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret") pod "global-pull-secret-syncer-6r8xd" (UID: "f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:59:44.885057 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:44.885015 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:45.002592 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:45.002552 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv2nv\" (UniqueName: \"kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv\") pod \"network-check-target-r92qp\" (UID: \"07721dc7-4474-4c7f-b037-878a06841f31\") " pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:45.002592 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:45.002600 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs\") pod \"network-metrics-daemon-qwxnw\" (UID: \"8dd93cd1-bc16-44a6-83fb-a0861aa19e26\") " pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:45.002823 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:45.002720 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:59:45.002823 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:45.002737 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:59:45.002823 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:45.002764 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:59:45.002823 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:45.002775 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs podName:8dd93cd1-bc16-44a6-83fb-a0861aa19e26 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:53.002757621 +0000 UTC m=+43.244752845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs") pod "network-metrics-daemon-qwxnw" (UID: "8dd93cd1-bc16-44a6-83fb-a0861aa19e26") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:59:45.002823 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:45.002776 2562 projected.go:196] Error preparing data for projected volume kube-api-access-lv2nv for pod openshift-network-diagnostics/network-check-target-r92qp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:59:45.002823 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:45.002824 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv podName:07721dc7-4474-4c7f-b037-878a06841f31 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:53.002807724 +0000 UTC m=+43.244802929 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lv2nv" (UniqueName: "kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv") pod "network-check-target-r92qp" (UID: "07721dc7-4474-4c7f-b037-878a06841f31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:59:45.035639 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:45.035601 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:45.219056 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:45.219020 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:45.318659 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:45.318607 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:45.927662 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:45.927630 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:46.005221 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:46.005188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:46.245395 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:46.245326 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:46.245395 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:46.245357 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:46.245611 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:46.245326 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:46.245611 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:46.245449 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r92qp" podUID="07721dc7-4474-4c7f-b037-878a06841f31" Jul 15 16:59:46.245611 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:46.245516 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6r8xd" podUID="f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca" Jul 15 16:59:46.245762 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:46.245645 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qwxnw" podUID="8dd93cd1-bc16-44a6-83fb-a0861aa19e26" Jul 15 16:59:48.246087 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:48.246053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:48.246660 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:48.246053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:48.246660 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:48.246181 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r92qp" podUID="07721dc7-4474-4c7f-b037-878a06841f31" Jul 15 16:59:48.246660 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:48.246213 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:48.246660 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:48.246299 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6r8xd" podUID="f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca" Jul 15 16:59:48.246660 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:48.246409 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qwxnw" podUID="8dd93cd1-bc16-44a6-83fb-a0861aa19e26" Jul 15 16:59:49.078095 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:49.078064 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:49.289856 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:49.289811 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:50.246380 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.246356 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:50.246492 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:50.246453 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6r8xd" podUID="f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca" Jul 15 16:59:50.246687 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.246669 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:50.246787 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.246773 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:50.246866 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:50.246780 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r92qp" podUID="07721dc7-4474-4c7f-b037-878a06841f31" Jul 15 16:59:50.246929 ip-10-0-135-132 kubenswrapper[2562]: E0715 16:59:50.246876 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qwxnw" podUID="8dd93cd1-bc16-44a6-83fb-a0861aa19e26" Jul 15 16:59:50.395970 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.395636 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w5h4s" event={"ID":"652c50d6-b43e-4f5c-8cc2-70d2155a9188","Type":"ContainerStarted","Data":"bae50e8adb8c1f894fac970467c4c03e5e23cbca029fdcfe85f534018914cab9"} Jul 15 16:59:50.397829 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.397802 2562 generic.go:356] "Generic (PLEG): container finished" podID="35c01559-3824-4e5c-bfea-71ead02ea8ed" containerID="f10e52b2458050c2948196b1c7da20451d8fb8d089a93aac645c544f160ddb44" exitCode=0 Jul 15 16:59:50.397963 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.397894 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxs9l" event={"ID":"35c01559-3824-4e5c-bfea-71ead02ea8ed","Type":"ContainerDied","Data":"f10e52b2458050c2948196b1c7da20451d8fb8d089a93aac645c544f160ddb44"} Jul 15 16:59:50.400382 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.400363 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal_1176b310d9fb52e4fefed71966d7b67a/setup/1.log" Jul 15 16:59:50.400507 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.400417 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" event={"ID":"1176b310d9fb52e4fefed71966d7b67a","Type":"ContainerStarted","Data":"67191f15eaeb366e41ac5a0d63a81df6daea9e4760096f41d87547dadd9833d5"} Jul 15 16:59:50.400691 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.400674 2562 scope.go:117] "RemoveContainer" containerID="ac7c5c90d03cfe895f05790407cecfb955a79207ce086f7d37fe2d95e435bfad" Jul 15 16:59:50.440799 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.440752 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w5h4s" podStartSLOduration=0.761716797 podStartE2EDuration="13.440736347s" totalImagesPullingTime="12.67901955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:50.440556663 +0000 UTC m=+40.682551891" watchObservedRunningTime="2026-07-15 16:59:50.440736347 +0000 UTC m=+40.682731572" Jul 15 16:59:50.759117 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.759038 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:50.897261 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:50.897213 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:51.073586 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:51.073557 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:51.224333 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:51.224298 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:51.405475 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:51.405394 2562 generic.go:356] "Generic (PLEG): container finished" podID="1176b310d9fb52e4fefed71966d7b67a" containerID="67191f15eaeb366e41ac5a0d63a81df6daea9e4760096f41d87547dadd9833d5" exitCode=0 Jul 15 16:59:51.405865 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:51.405468 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" event={"ID":"1176b310d9fb52e4fefed71966d7b67a","Type":"ContainerDied","Data":"67191f15eaeb366e41ac5a0d63a81df6daea9e4760096f41d87547dadd9833d5"} Jul 15 16:59:51.405865 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:51.405571 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" event={"ID":"1176b310d9fb52e4fefed71966d7b67a","Type":"ContainerStarted","Data":"7657fa1838418433f2444836d54912bd89cfd6eb6f1b76d08991e192f39a2fd0"} Jul 15 16:59:51.850265 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:51.850228 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:52.029079 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.029041 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:52.138001 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.137926 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-132.ec2.internal" event="NodeReady" Jul 15 16:59:52.138156 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.138082 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 16:59:52.227906 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.227819 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-132.ec2.internal" podStartSLOduration=40.227794779 podStartE2EDuration="40.227794779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:51.459143412 +0000 UTC m=+41.701138637" watchObservedRunningTime="2026-07-15 16:59:52.227794779 +0000 UTC m=+42.469790006" Jul 15 16:59:52.228119 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.228093 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bbtx8"] Jul 15 16:59:52.241339 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.241313 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rjhcq"] Jul 15 16:59:52.241455 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.241437 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.247125 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.247105 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 16:59:52.247125 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.247117 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 16:59:52.247300 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.247148 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 16:59:52.249137 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.249119 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 16:59:52.249746 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.249441 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2tcfp\"" Jul 15 16:59:52.251529 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.251510 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rjhcq" Jul 15 16:59:52.254897 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.254877 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rjhcq"] Jul 15 16:59:52.254981 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.254973 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bbtx8"] Jul 15 16:59:52.255075 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.255062 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:52.255136 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.255071 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:52.255136 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.255071 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:52.259121 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.258689 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 16:59:52.259121 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.258753 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kt5f5\"" Jul 15 16:59:52.261406 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.261387 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 16:59:52.261641 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.261628 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 16:59:52.261821 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.261800 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f7t54\"" Jul 15 16:59:52.262572 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.262554 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 16:59:52.262675 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.262602 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 16:59:52.262675 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.262627 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 16:59:52.262675 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.262633 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tns8n\"" Jul 15 16:59:52.262675 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.262652 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 16:59:52.283524 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.283501 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-td2hl"] Jul 15 16:59:52.290094 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.290076 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.299029 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.299005 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 16:59:52.299118 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.299059 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5hxs5\"" Jul 15 16:59:52.299179 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.299145 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 16:59:52.317778 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.317753 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-td2hl"] Jul 15 16:59:52.358364 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.358341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8prr\" (UniqueName: \"kubernetes.io/projected/6283c946-6d33-4e68-9bd5-94ff19b00c31-kube-api-access-t8prr\") pod \"ingress-canary-rjhcq\" (UID: \"6283c946-6d33-4e68-9bd5-94ff19b00c31\") " pod="openshift-ingress-canary/ingress-canary-rjhcq" Jul 15 16:59:52.358498 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.358382 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ba4bd8c-b016-4515-865b-6977c37af13e-crio-socket\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.358498 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.358405 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba4bd8c-b016-4515-865b-6977c37af13e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.358498 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.358452 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ba4bd8c-b016-4515-865b-6977c37af13e-data-volume\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.358498 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.358475 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6283c946-6d33-4e68-9bd5-94ff19b00c31-cert\") pod \"ingress-canary-rjhcq\" (UID: \"6283c946-6d33-4e68-9bd5-94ff19b00c31\") " pod="openshift-ingress-canary/ingress-canary-rjhcq" Jul 15 16:59:52.358498 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.358494 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ba4bd8c-b016-4515-865b-6977c37af13e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.358668 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.358510 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8h6d\" (UniqueName: \"kubernetes.io/projected/5ba4bd8c-b016-4515-865b-6977c37af13e-kube-api-access-j8h6d\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.410056 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.409988 2562 generic.go:356] "Generic (PLEG): container finished" podID="35c01559-3824-4e5c-bfea-71ead02ea8ed" containerID="f0655f19b8f3717b0d32cfda0b42facb3a7fa105fafd038a6be529ca6f0563b1" exitCode=0 Jul 15 16:59:52.410056 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.410035 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxs9l" event={"ID":"35c01559-3824-4e5c-bfea-71ead02ea8ed","Type":"ContainerDied","Data":"f0655f19b8f3717b0d32cfda0b42facb3a7fa105fafd038a6be529ca6f0563b1"} Jul 15 16:59:52.459262 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459236 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6283c946-6d33-4e68-9bd5-94ff19b00c31-cert\") pod \"ingress-canary-rjhcq\" (UID: \"6283c946-6d33-4e68-9bd5-94ff19b00c31\") " pod="openshift-ingress-canary/ingress-canary-rjhcq" Jul 15 16:59:52.459369 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459270 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqlp8\" (UniqueName: \"kubernetes.io/projected/c86c79e5-143a-4be2-b192-8192c6dedc20-kube-api-access-tqlp8\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.459369 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ba4bd8c-b016-4515-865b-6977c37af13e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.459369 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459308 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8h6d\" (UniqueName: \"kubernetes.io/projected/5ba4bd8c-b016-4515-865b-6977c37af13e-kube-api-access-j8h6d\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.459369 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459330 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c86c79e5-143a-4be2-b192-8192c6dedc20-config-volume\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.459369 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8prr\" (UniqueName: \"kubernetes.io/projected/6283c946-6d33-4e68-9bd5-94ff19b00c31-kube-api-access-t8prr\") pod \"ingress-canary-rjhcq\" (UID: \"6283c946-6d33-4e68-9bd5-94ff19b00c31\") " pod="openshift-ingress-canary/ingress-canary-rjhcq" Jul 15 16:59:52.459619 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459372 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c86c79e5-143a-4be2-b192-8192c6dedc20-tmp-dir\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.459619 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459397 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ba4bd8c-b016-4515-865b-6977c37af13e-crio-socket\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.459619 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba4bd8c-b016-4515-865b-6977c37af13e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.459619 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ba4bd8c-b016-4515-865b-6977c37af13e-data-volume\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.459619 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c86c79e5-143a-4be2-b192-8192c6dedc20-metrics-tls\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.459895 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459616 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ba4bd8c-b016-4515-865b-6977c37af13e-crio-socket\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.459895 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.459738 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ba4bd8c-b016-4515-865b-6977c37af13e-data-volume\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.460028 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.460006 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ba4bd8c-b016-4515-865b-6977c37af13e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.463665 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.463634 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba4bd8c-b016-4515-865b-6977c37af13e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.463769 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.463717 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6283c946-6d33-4e68-9bd5-94ff19b00c31-cert\") pod \"ingress-canary-rjhcq\" (UID: \"6283c946-6d33-4e68-9bd5-94ff19b00c31\") " pod="openshift-ingress-canary/ingress-canary-rjhcq" Jul 15 16:59:52.469362 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.469331 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8h6d\" (UniqueName: \"kubernetes.io/projected/5ba4bd8c-b016-4515-865b-6977c37af13e-kube-api-access-j8h6d\") pod \"insights-runtime-extractor-bbtx8\" (UID: \"5ba4bd8c-b016-4515-865b-6977c37af13e\") " pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.469454 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.469367 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8prr\" (UniqueName: \"kubernetes.io/projected/6283c946-6d33-4e68-9bd5-94ff19b00c31-kube-api-access-t8prr\") pod \"ingress-canary-rjhcq\" (UID: \"6283c946-6d33-4e68-9bd5-94ff19b00c31\") " pod="openshift-ingress-canary/ingress-canary-rjhcq" Jul 15 16:59:52.471624 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.471598 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:52.551376 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.551345 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bbtx8" Jul 15 16:59:52.560646 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.560621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqlp8\" (UniqueName: \"kubernetes.io/projected/c86c79e5-143a-4be2-b192-8192c6dedc20-kube-api-access-tqlp8\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.560706 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.560690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c86c79e5-143a-4be2-b192-8192c6dedc20-config-volume\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.560796 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.560772 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c86c79e5-143a-4be2-b192-8192c6dedc20-tmp-dir\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.560918 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.560901 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c86c79e5-143a-4be2-b192-8192c6dedc20-metrics-tls\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.561144 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.561121 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c86c79e5-143a-4be2-b192-8192c6dedc20-tmp-dir\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.561277 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.561258 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c86c79e5-143a-4be2-b192-8192c6dedc20-config-volume\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.561317 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.561266 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rjhcq" Jul 15 16:59:52.563269 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.563252 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c86c79e5-143a-4be2-b192-8192c6dedc20-metrics-tls\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.569610 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.569592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqlp8\" (UniqueName: \"kubernetes.io/projected/c86c79e5-143a-4be2-b192-8192c6dedc20-kube-api-access-tqlp8\") pod \"dns-default-td2hl\" (UID: \"c86c79e5-143a-4be2-b192-8192c6dedc20\") " pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.571739 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.571717 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:52.597380 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.597352 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-td2hl" Jul 15 16:59:52.796279 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.796082 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rjhcq"] Jul 15 16:59:52.802114 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:52.802086 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6283c946_6d33_4e68_9bd5_94ff19b00c31.slice/crio-5e14a8a661d26ede88dea5521a430018e8d076b1bb1cbc01b876e773e7bf4044 WatchSource:0}: Error finding container 5e14a8a661d26ede88dea5521a430018e8d076b1bb1cbc01b876e773e7bf4044: Status 404 returned error can't find the container with id 5e14a8a661d26ede88dea5521a430018e8d076b1bb1cbc01b876e773e7bf4044 Jul 15 16:59:52.806882 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.806851 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bbtx8"] Jul 15 16:59:52.817920 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:52.817900 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba4bd8c_b016_4515_865b_6977c37af13e.slice/crio-4d8bce787ece978993f9710a5bbb5787830697198feb40a628c8a996ac1b0178 WatchSource:0}: Error finding container 4d8bce787ece978993f9710a5bbb5787830697198feb40a628c8a996ac1b0178: Status 404 returned error can't find the container with id 4d8bce787ece978993f9710a5bbb5787830697198feb40a628c8a996ac1b0178 Jul 15 16:59:52.828378 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.828358 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-td2hl"] Jul 15 16:59:52.830981 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:52.830961 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc86c79e5_143a_4be2_b192_8192c6dedc20.slice/crio-8ba512fb86cf8bf0563b4396219ccd66a84669f89d4f5db620febfc48358e109 WatchSource:0}: Error finding container 8ba512fb86cf8bf0563b4396219ccd66a84669f89d4f5db620febfc48358e109: Status 404 returned error can't find the container with id 8ba512fb86cf8bf0563b4396219ccd66a84669f89d4f5db620febfc48358e109 Jul 15 16:59:52.863448 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.863415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:52.866823 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.866806 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca-original-pull-secret\") pod \"global-pull-secret-syncer-6r8xd\" (UID: \"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca\") " pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:52.870744 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.870727 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6r8xd" Jul 15 16:59:52.942474 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:52.942388 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:53.020712 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.020681 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6r8xd"] Jul 15 16:59:53.024746 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:53.024718 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c52b79_c004_44fb_84b8_ee8c8a9ca5ca.slice/crio-4290e69b6740b5cf45b1d1e058f47b83ab6810fc3be03927596bc17e114fe50d WatchSource:0}: Error finding container 4290e69b6740b5cf45b1d1e058f47b83ab6810fc3be03927596bc17e114fe50d: Status 404 returned error can't find the container with id 4290e69b6740b5cf45b1d1e058f47b83ab6810fc3be03927596bc17e114fe50d Jul 15 16:59:53.064693 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.064668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv2nv\" (UniqueName: \"kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv\") pod \"network-check-target-r92qp\" (UID: \"07721dc7-4474-4c7f-b037-878a06841f31\") " pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:53.064802 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.064703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs\") pod \"network-metrics-daemon-qwxnw\" (UID: \"8dd93cd1-bc16-44a6-83fb-a0861aa19e26\") " pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:53.067243 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.067214 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd93cd1-bc16-44a6-83fb-a0861aa19e26-metrics-certs\") pod \"network-metrics-daemon-qwxnw\" (UID: \"8dd93cd1-bc16-44a6-83fb-a0861aa19e26\") " pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:53.067434 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.067416 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv2nv\" (UniqueName: \"kubernetes.io/projected/07721dc7-4474-4c7f-b037-878a06841f31-kube-api-access-lv2nv\") pod \"network-check-target-r92qp\" (UID: \"07721dc7-4474-4c7f-b037-878a06841f31\") " pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:53.147786 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.147754 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:53.175704 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.175676 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qwxnw" Jul 15 16:59:53.179810 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.179791 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 16:59:53.268473 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.268236 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:53.341718 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.341693 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qwxnw"] Jul 15 16:59:53.344971 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:53.344945 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd93cd1_bc16_44a6_83fb_a0861aa19e26.slice/crio-7103db1660683acc2170d4c210ee0695afb26715a04d91d8c75d55f1e5574ddd WatchSource:0}: Error finding container 7103db1660683acc2170d4c210ee0695afb26715a04d91d8c75d55f1e5574ddd: Status 404 returned error can't find the container with id 7103db1660683acc2170d4c210ee0695afb26715a04d91d8c75d55f1e5574ddd Jul 15 16:59:53.359715 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.359690 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r92qp"] Jul 15 16:59:53.361888 ip-10-0-135-132 kubenswrapper[2562]: W0715 16:59:53.361865 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07721dc7_4474_4c7f_b037_878a06841f31.slice/crio-310440bf6f81329ce65c34075e77454e164b1ba96130d722f93660cc037047f8 WatchSource:0}: Error finding container 310440bf6f81329ce65c34075e77454e164b1ba96130d722f93660cc037047f8: Status 404 returned error can't find the container with id 310440bf6f81329ce65c34075e77454e164b1ba96130d722f93660cc037047f8 Jul 15 16:59:53.414544 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.414511 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qwxnw" event={"ID":"8dd93cd1-bc16-44a6-83fb-a0861aa19e26","Type":"ContainerStarted","Data":"7103db1660683acc2170d4c210ee0695afb26715a04d91d8c75d55f1e5574ddd"} Jul 15 16:59:53.415726 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.415699 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r92qp" event={"ID":"07721dc7-4474-4c7f-b037-878a06841f31","Type":"ContainerStarted","Data":"310440bf6f81329ce65c34075e77454e164b1ba96130d722f93660cc037047f8"} Jul 15 16:59:53.416995 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.416970 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6r8xd" event={"ID":"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca","Type":"ContainerStarted","Data":"4290e69b6740b5cf45b1d1e058f47b83ab6810fc3be03927596bc17e114fe50d"} Jul 15 16:59:53.418416 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.418387 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rjhcq" event={"ID":"6283c946-6d33-4e68-9bd5-94ff19b00c31","Type":"ContainerStarted","Data":"5e14a8a661d26ede88dea5521a430018e8d076b1bb1cbc01b876e773e7bf4044"} Jul 15 16:59:53.419545 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.419511 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bbtx8" event={"ID":"5ba4bd8c-b016-4515-865b-6977c37af13e","Type":"ContainerStarted","Data":"4d8bce787ece978993f9710a5bbb5787830697198feb40a628c8a996ac1b0178"} Jul 15 16:59:53.420653 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:53.420632 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-td2hl" event={"ID":"c86c79e5-143a-4be2-b192-8192c6dedc20","Type":"ContainerStarted","Data":"8ba512fb86cf8bf0563b4396219ccd66a84669f89d4f5db620febfc48358e109"} Jul 15 16:59:54.066356 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:54.066321 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:54.147514 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:54.147429 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:54.431418 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:54.430438 2562 generic.go:356] "Generic (PLEG): container finished" podID="35c01559-3824-4e5c-bfea-71ead02ea8ed" containerID="09f2279b4ed8637cf027fb4177ad5cd0fd305773f7775b2c84c3f856be2cf48e" exitCode=0 Jul 15 16:59:54.431418 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:54.430670 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxs9l" event={"ID":"35c01559-3824-4e5c-bfea-71ead02ea8ed","Type":"ContainerDied","Data":"09f2279b4ed8637cf027fb4177ad5cd0fd305773f7775b2c84c3f856be2cf48e"} Jul 15 16:59:54.505785 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:54.505192 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:54.573641 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:54.573604 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:55.697335 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:55.697258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:55.809741 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:55.809630 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:56.444785 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:56.444663 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-td2hl" event={"ID":"c86c79e5-143a-4be2-b192-8192c6dedc20","Type":"ContainerStarted","Data":"7e2639fb22ed7091a2a0f714af82b9c4b1f6e7051351809dbe7e9e0cbbc38137"} Jul 15 16:59:56.446729 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:56.446692 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qwxnw" event={"ID":"8dd93cd1-bc16-44a6-83fb-a0861aa19e26","Type":"ContainerStarted","Data":"b4d963ace68d534e5552bacdd9ca140a75c2609dc2953815701cc8ec39cf3565"} Jul 15 16:59:56.840250 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:56.840167 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:56.950194 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:56.950163 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:57.832654 ip-10-0-135-132 kubenswrapper[2562]: I0715 16:59:57.832593 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:00.296598 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.296553 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:00.741944 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.741909 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:00.768500 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.768474 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr"] Jul 15 17:00:00.911215 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.911177 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:00.981083 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.981051 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr"] Jul 15 17:00:00.981083 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.981089 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-lpq5l"] Jul 15 17:00:00.981254 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.981169 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:00.984463 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.984446 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 15 17:00:00.984740 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.984722 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 15 17:00:00.986204 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.986188 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 17:00:00.987961 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.987942 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bnxsc\"" Jul 15 17:00:00.988676 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.988661 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 17:00:00.989566 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:00.989546 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 17:00:01.001335 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.001290 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-lpq5l"] Jul 15 17:00:01.001335 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.001323 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vvmkg"] Jul 15 17:00:01.001444 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.001395 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.004463 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.004448 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 15 17:00:01.004633 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.004621 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-wlmgn\"" Jul 15 17:00:01.005032 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.005018 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 15 17:00:01.005169 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.005156 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 15 17:00:01.025992 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.025972 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.028739 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.028721 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 17:00:01.028868 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.028766 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 17:00:01.029062 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.029042 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 17:00:01.029062 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.029056 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rnszz\"" Jul 15 17:00:01.120427 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.120390 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:01.126256 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126229 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-textfile\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.126367 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126280 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8506444a-b35c-474f-84e0-e3bceb00271a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.126367 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7adfd351-5cdc-43c3-8684-553480b8ab6f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.126367 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126331 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7adfd351-5cdc-43c3-8684-553480b8ab6f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.126367 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126356 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-tls\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126379 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klrsl\" (UniqueName: \"kubernetes.io/projected/7adfd351-5cdc-43c3-8684-553480b8ab6f-kube-api-access-klrsl\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126399 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-accelerators-collector-config\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126417 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8506444a-b35c-474f-84e0-e3bceb00271a-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e14736e-6d94-4699-89e4-3a05998d1afe-sys\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7adfd351-5cdc-43c3-8684-553480b8ab6f-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126483 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e14736e-6d94-4699-89e4-3a05998d1afe-root\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126500 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5xkf\" (UniqueName: \"kubernetes.io/projected/8506444a-b35c-474f-84e0-e3bceb00271a-kube-api-access-p5xkf\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126527 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8506444a-b35c-474f-84e0-e3bceb00271a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126548 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-wtmp\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65d2j\" (UniqueName: \"kubernetes.io/projected/7e14736e-6d94-4699-89e4-3a05998d1afe-kube-api-access-65d2j\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126593 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.126612 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126612 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8506444a-b35c-474f-84e0-e3bceb00271a-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.127381 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e14736e-6d94-4699-89e4-3a05998d1afe-metrics-client-ca\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.127381 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.126648 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8506444a-b35c-474f-84e0-e3bceb00271a-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.227782 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.227753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-accelerators-collector-config\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.227782 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.227789 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8506444a-b35c-474f-84e0-e3bceb00271a-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.228031 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.227811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e14736e-6d94-4699-89e4-3a05998d1afe-sys\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.228031 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.227950 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7adfd351-5cdc-43c3-8684-553480b8ab6f-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.228031 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228003 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e14736e-6d94-4699-89e4-3a05998d1afe-root\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.228151 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228029 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5xkf\" (UniqueName: \"kubernetes.io/projected/8506444a-b35c-474f-84e0-e3bceb00271a-kube-api-access-p5xkf\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.228151 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228067 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8506444a-b35c-474f-84e0-e3bceb00271a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.228151 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228104 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-wtmp\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.228151 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65d2j\" (UniqueName: \"kubernetes.io/projected/7e14736e-6d94-4699-89e4-3a05998d1afe-kube-api-access-65d2j\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.228337 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.228337 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8506444a-b35c-474f-84e0-e3bceb00271a-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.228337 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228213 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8506444a-b35c-474f-84e0-e3bceb00271a-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.228337 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228229 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e14736e-6d94-4699-89e4-3a05998d1afe-metrics-client-ca\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.228337 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228263 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8506444a-b35c-474f-84e0-e3bceb00271a-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.228337 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-textfile\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.228337 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228334 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8506444a-b35c-474f-84e0-e3bceb00271a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.228605 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228369 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7adfd351-5cdc-43c3-8684-553480b8ab6f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.228605 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228401 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7adfd351-5cdc-43c3-8684-553480b8ab6f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.228605 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228409 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-accelerators-collector-config\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.228605 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-tls\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.228605 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:00:01.228496 2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 15 17:00:01.228605 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:00:01.228564 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7adfd351-5cdc-43c3-8684-553480b8ab6f-openshift-state-metrics-tls podName:7adfd351-5cdc-43c3-8684-553480b8ab6f nodeName:}" failed. No retries permitted until 2026-07-15 17:00:01.728537462 +0000 UTC m=+51.970532675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7adfd351-5cdc-43c3-8684-553480b8ab6f-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-npdxr" (UID: "7adfd351-5cdc-43c3-8684-553480b8ab6f") : secret "openshift-state-metrics-tls" not found Jul 15 17:00:01.228872 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228643 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e14736e-6d94-4699-89e4-3a05998d1afe-metrics-client-ca\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.228872 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228698 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e14736e-6d94-4699-89e4-3a05998d1afe-sys\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.230004 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.228495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klrsl\" (UniqueName: \"kubernetes.io/projected/7adfd351-5cdc-43c3-8684-553480b8ab6f-kube-api-access-klrsl\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.230004 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.229311 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-wtmp\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.230004 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.229368 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e14736e-6d94-4699-89e4-3a05998d1afe-root\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.230004 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.229405 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-textfile\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.230004 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.229406 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7adfd351-5cdc-43c3-8684-553480b8ab6f-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.230525 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.230156 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8506444a-b35c-474f-84e0-e3bceb00271a-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.230525 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.230290 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8506444a-b35c-474f-84e0-e3bceb00271a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.232307 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.232283 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7adfd351-5cdc-43c3-8684-553480b8ab6f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.232422 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.232290 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-tls\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.232751 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.232723 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8506444a-b35c-474f-84e0-e3bceb00271a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.233588 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.233565 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8506444a-b35c-474f-84e0-e3bceb00271a-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.234070 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.234050 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e14736e-6d94-4699-89e4-3a05998d1afe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.244454 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.244434 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klrsl\" (UniqueName: \"kubernetes.io/projected/7adfd351-5cdc-43c3-8684-553480b8ab6f-kube-api-access-klrsl\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.245817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.245795 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5xkf\" (UniqueName: \"kubernetes.io/projected/8506444a-b35c-474f-84e0-e3bceb00271a-kube-api-access-p5xkf\") pod \"kube-state-metrics-785778ff74-lpq5l\" (UID: \"8506444a-b35c-474f-84e0-e3bceb00271a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.248364 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.248344 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65d2j\" (UniqueName: \"kubernetes.io/projected/7e14736e-6d94-4699-89e4-3a05998d1afe-kube-api-access-65d2j\") pod \"node-exporter-vvmkg\" (UID: \"7e14736e-6d94-4699-89e4-3a05998d1afe\") " pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.309059 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.308996 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" Jul 15 17:00:01.338016 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.337827 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vvmkg" Jul 15 17:00:01.450520 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.450450 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:01.462153 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.462123 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qwxnw" event={"ID":"8dd93cd1-bc16-44a6-83fb-a0861aa19e26","Type":"ContainerStarted","Data":"08ec2d0252ce7d5a9c5d4790e8cf76ba56dace79a689be6aa38a7e937d29771a"} Jul 15 17:00:01.733603 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.733564 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7adfd351-5cdc-43c3-8684-553480b8ab6f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.736358 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.736334 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7adfd351-5cdc-43c3-8684-553480b8ab6f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-npdxr\" (UID: \"7adfd351-5cdc-43c3-8684-553480b8ab6f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.810237 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.810183 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qwxnw" podStartSLOduration=22.065313616 podStartE2EDuration="24.810160674s" totalImagesPullingTime="2.744847058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:01.500008302 +0000 UTC m=+51.742003533" watchObservedRunningTime="2026-07-15 17:00:01.810160674 +0000 UTC m=+52.052155894" Jul 15 17:00:01.810746 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.810721 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 17:00:01.819462 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.819440 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:01.836982 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.836951 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.842935 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.842900 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 17:00:01.843198 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.843180 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nj669\"" Jul 15 17:00:01.843436 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.843422 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 17:00:01.843967 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.843771 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 17:00:01.843967 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.843877 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 17:00:01.844118 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.844008 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 17:00:01.844118 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.844072 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 17:00:01.844222 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.844202 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 17:00:01.844395 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.844273 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 17:00:01.844495 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.844482 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 17:00:01.844627 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.844598 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 17:00:01.890670 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.890639 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" Jul 15 17:00:01.935330 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935300 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-config-out\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.935494 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935339 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.935494 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.935494 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935396 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.935494 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.935714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935516 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-config-volume\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.935714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935539 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.935714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935602 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.935714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.935714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935654 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-699bv\" (UniqueName: \"kubernetes.io/projected/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-kube-api-access-699bv\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.935714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935698 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.936021 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935728 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-tls-assets\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:01.936021 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:01.935745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-web-config\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036176 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036176 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036176 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036168 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699bv\" (UniqueName: \"kubernetes.io/projected/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-kube-api-access-699bv\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036408 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036408 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036227 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-tls-assets\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036408 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-web-config\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036408 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036286 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-config-out\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036408 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036308 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036408 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036408 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036717 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036431 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036717 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036460 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-config-volume\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036717 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036481 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036879 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036800 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.036940 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.036913 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.038549 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.038247 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.040299 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.040233 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.040414 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.040376 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.040758 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.040730 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:02.041284 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.041257 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-config-out\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.041386 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.041280 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.041453 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.041389 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-web-config\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.041570 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.041546 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.042035 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.041988 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-config-volume\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.042138 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.042091 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-tls-assets\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.042435 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.042408 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.060044 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.060019 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-699bv\" (UniqueName: \"kubernetes.io/projected/87a18837-b8d6-47ce-89d7-bc3fbfc1cc36-kube-api-access-699bv\") pod \"alertmanager-main-0\" (UID: \"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:02.149177 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:02.149144 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 17:00:03.742976 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:03.742940 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:03.918851 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:03.918809 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-749db7cfc4-tw8m9"] Jul 15 17:00:03.952398 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:03.952363 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:03.963424 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:03.963386 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 15 17:00:03.963552 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:03.963507 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hqpv6\"" Jul 15 17:00:03.963658 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:03.963631 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 15 17:00:03.964794 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:03.964775 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 15 17:00:03.964903 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:03.964867 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ans7vqnj2f9qc\"" Jul 15 17:00:03.967302 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:03.967282 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 15 17:00:03.968815 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:03.968801 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 15 17:00:03.979571 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:03.979402 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-749db7cfc4-tw8m9"] Jul 15 17:00:04.005791 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.005714 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:04.049779 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.049745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-grpc-tls\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.049975 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.049808 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1553e479-bbd5-40d4-b7b1-060572599bb9-metrics-client-ca\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.049975 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.049854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d245n\" (UniqueName: \"kubernetes.io/projected/1553e479-bbd5-40d4-b7b1-060572599bb9-kube-api-access-d245n\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.049975 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.049934 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.049975 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.049966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-tls\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.050120 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.049993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.050120 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.050019 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.050120 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.050044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.150661 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.150621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.150808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.150695 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-grpc-tls\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.150808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.150732 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1553e479-bbd5-40d4-b7b1-060572599bb9-metrics-client-ca\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.150808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.150757 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d245n\" (UniqueName: \"kubernetes.io/projected/1553e479-bbd5-40d4-b7b1-060572599bb9-kube-api-access-d245n\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.150808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.150805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.151052 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.150867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-tls\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.151052 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.150907 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.151052 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.150936 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.151561 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.151531 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1553e479-bbd5-40d4-b7b1-060572599bb9-metrics-client-ca\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.153978 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.153946 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.154143 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.154101 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.154359 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.154338 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.154474 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.154377 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-grpc-tls\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.155078 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.155053 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.155191 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.155141 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1553e479-bbd5-40d4-b7b1-060572599bb9-secret-thanos-querier-tls\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.162464 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.162440 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d245n\" (UniqueName: \"kubernetes.io/projected/1553e479-bbd5-40d4-b7b1-060572599bb9-kube-api-access-d245n\") pod \"thanos-querier-749db7cfc4-tw8m9\" (UID: \"1553e479-bbd5-40d4-b7b1-060572599bb9\") " pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.167589 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.167557 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:04.262562 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.262461 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:04.347342 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.347308 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vclsh" Jul 15 17:00:04.502879 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.502777 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:04.683894 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.683861 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:04.876161 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:04.876126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:05.308528 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.308445 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:05.545752 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.545627 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:05.554592 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.554567 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg"] Jul 15 17:00:05.576272 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.576234 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg"] Jul 15 17:00:05.576418 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.576358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" Jul 15 17:00:05.585416 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.585396 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 15 17:00:05.592023 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.592004 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2fgfg\"" Jul 15 17:00:05.765590 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.765559 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a7be7b5-4e96-4075-9880-ed9aed255eab-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-rctbg\" (UID: \"9a7be7b5-4e96-4075-9880-ed9aed255eab\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" Jul 15 17:00:05.765743 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.765624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg5fr\" (UniqueName: \"kubernetes.io/projected/9a7be7b5-4e96-4075-9880-ed9aed255eab-kube-api-access-xg5fr\") pod \"monitoring-plugin-5f68df7f58-rctbg\" (UID: \"9a7be7b5-4e96-4075-9880-ed9aed255eab\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" Jul 15 17:00:05.872355 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.872326 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a7be7b5-4e96-4075-9880-ed9aed255eab-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-rctbg\" (UID: \"9a7be7b5-4e96-4075-9880-ed9aed255eab\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" Jul 15 17:00:05.872499 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.872416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg5fr\" (UniqueName: \"kubernetes.io/projected/9a7be7b5-4e96-4075-9880-ed9aed255eab-kube-api-access-xg5fr\") pod \"monitoring-plugin-5f68df7f58-rctbg\" (UID: \"9a7be7b5-4e96-4075-9880-ed9aed255eab\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" Jul 15 17:00:05.876446 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.876389 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a7be7b5-4e96-4075-9880-ed9aed255eab-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-rctbg\" (UID: \"9a7be7b5-4e96-4075-9880-ed9aed255eab\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" Jul 15 17:00:05.883724 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.883212 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-87dcf" Jul 15 17:00:05.884804 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.884552 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-87dcf" Jul 15 17:00:05.914910 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.914880 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg5fr\" (UniqueName: \"kubernetes.io/projected/9a7be7b5-4e96-4075-9880-ed9aed255eab-kube-api-access-xg5fr\") pod \"monitoring-plugin-5f68df7f58-rctbg\" (UID: \"9a7be7b5-4e96-4075-9880-ed9aed255eab\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" Jul 15 17:00:05.935745 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:05.935665 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:06.060079 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.059609 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6c4b99847c-2vf5v"] Jul 15 17:00:06.080942 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.080685 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr"] Jul 15 17:00:06.080942 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.080891 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.094826 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.094803 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 17:00:06.095045 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.095031 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 17:00:06.095573 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.095455 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 17:00:06.096126 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.095963 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 17:00:06.096126 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.095967 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 17:00:06.096540 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.096243 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hngd8\"" Jul 15 17:00:06.112740 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.112718 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 17:00:06.120783 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.120245 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c4b99847c-2vf5v"] Jul 15 17:00:06.121961 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.121364 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-749db7cfc4-tw8m9"] Jul 15 17:00:06.129911 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.129791 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:06.175575 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.175546 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7n7z\" (UniqueName: \"kubernetes.io/projected/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-kube-api-access-v7n7z\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.175575 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.175583 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-serving-certs-ca-bundle\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.175808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.175614 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.175808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.175677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-telemeter-client-tls\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.175808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.175704 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.175808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.175729 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-federate-client-tls\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.175808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.175753 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-secret-telemeter-client\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.175808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.175785 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-metrics-client-ca\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.187524 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.187492 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" Jul 15 17:00:06.267588 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.267551 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:06.276516 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.276490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-telemeter-client-tls\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.276654 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.276522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.276654 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.276543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-federate-client-tls\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.276654 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.276567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-secret-telemeter-client\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.276654 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.276597 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-metrics-client-ca\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.276882 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.276654 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7n7z\" (UniqueName: \"kubernetes.io/projected/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-kube-api-access-v7n7z\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.276882 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.276683 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-serving-certs-ca-bundle\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.276882 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.276704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.277621 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.277572 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-serving-certs-ca-bundle\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.277884 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.277800 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-metrics-client-ca\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.278056 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.278007 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.279672 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.279632 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.279827 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.279808 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-secret-telemeter-client\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.279933 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.279915 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-federate-client-tls\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.280028 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.280009 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-telemeter-client-tls\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.292315 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:00:06.292271 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7adfd351_5cdc_43c3_8684_553480b8ab6f.slice/crio-54fa31166c20f7a027978ade72951060062153d0bca5c1395ccbfd3ea359d114 WatchSource:0}: Error finding container 54fa31166c20f7a027978ade72951060062153d0bca5c1395ccbfd3ea359d114: Status 404 returned error can't find the container with id 54fa31166c20f7a027978ade72951060062153d0bca5c1395ccbfd3ea359d114 Jul 15 17:00:06.296509 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.296477 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7n7z\" (UniqueName: \"kubernetes.io/projected/7ffce7c9-7583-4e7a-a194-8b0827bc2ee1-kube-api-access-v7n7z\") pod \"telemeter-client-6c4b99847c-2vf5v\" (UID: \"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1\") " pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.315991 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.315967 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-lpq5l"] Jul 15 17:00:06.327278 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.327257 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 17:00:06.329645 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:00:06.329624 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8506444a_b35c_474f_84e0_e3bceb00271a.slice/crio-c34118621b3658ae59a87891cf92138c5152ec41f0d855e6929d67fba37b880e WatchSource:0}: Error finding container c34118621b3658ae59a87891cf92138c5152ec41f0d855e6929d67fba37b880e: Status 404 returned error can't find the container with id c34118621b3658ae59a87891cf92138c5152ec41f0d855e6929d67fba37b880e Jul 15 17:00:06.330986 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:00:06.330958 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87a18837_b8d6_47ce_89d7_bc3fbfc1cc36.slice/crio-7a967739f6ce400a069cd82306e747045bed4485d2000052b5abca768c58ead6 WatchSource:0}: Error finding container 7a967739f6ce400a069cd82306e747045bed4485d2000052b5abca768c58ead6: Status 404 returned error can't find the container with id 7a967739f6ce400a069cd82306e747045bed4485d2000052b5abca768c58ead6 Jul 15 17:00:06.411777 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.411751 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" Jul 15 17:00:06.469866 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.469800 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg"] Jul 15 17:00:06.473580 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:00:06.473545 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a7be7b5_4e96_4075_9880_ed9aed255eab.slice/crio-690cc4194667d3db677ff92bfa2dfcccc4b8c66ab6ab54c64053d762989d185f WatchSource:0}: Error finding container 690cc4194667d3db677ff92bfa2dfcccc4b8c66ab6ab54c64053d762989d185f: Status 404 returned error can't find the container with id 690cc4194667d3db677ff92bfa2dfcccc4b8c66ab6ab54c64053d762989d185f Jul 15 17:00:06.484328 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.483422 2562 generic.go:356] "Generic (PLEG): container finished" podID="35c01559-3824-4e5c-bfea-71ead02ea8ed" containerID="54bc6f294c713c2d62574d971fdff99d34c4929760e693995e8898dffb8c390b" exitCode=0 Jul 15 17:00:06.484328 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.483503 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxs9l" event={"ID":"35c01559-3824-4e5c-bfea-71ead02ea8ed","Type":"ContainerDied","Data":"54bc6f294c713c2d62574d971fdff99d34c4929760e693995e8898dffb8c390b"} Jul 15 17:00:06.494275 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.494166 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r92qp" event={"ID":"07721dc7-4474-4c7f-b037-878a06841f31","Type":"ContainerStarted","Data":"1ba569c3df769c9aa9f7ab0017fc0f9ed05343ba1d3cb2c5105127dc9eaef8cf"} Jul 15 17:00:06.498265 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.498194 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6r8xd" event={"ID":"f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca","Type":"ContainerStarted","Data":"0afcf05051ba2a9a1c1952249dffd54e1d551dbf846316c463b6084e36bea503"} Jul 15 17:00:06.500505 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.500461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" event={"ID":"1553e479-bbd5-40d4-b7b1-060572599bb9","Type":"ContainerStarted","Data":"d57cd954172aac540bc8c06322ee2b08d83e575dcd64d2662c3cafe7cef3a7a4"} Jul 15 17:00:06.501649 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.501574 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 17:00:06.504331 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.504232 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" event={"ID":"7adfd351-5cdc-43c3-8684-553480b8ab6f","Type":"ContainerStarted","Data":"ffc4070d3e199bcfa821aa00db8c7a45cf2220de59c440fdbf97d1b5c7c544d3"} Jul 15 17:00:06.504331 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.504291 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" event={"ID":"7adfd351-5cdc-43c3-8684-553480b8ab6f","Type":"ContainerStarted","Data":"54fa31166c20f7a027978ade72951060062153d0bca5c1395ccbfd3ea359d114"} Jul 15 17:00:06.506405 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.506361 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rjhcq" event={"ID":"6283c946-6d33-4e68-9bd5-94ff19b00c31","Type":"ContainerStarted","Data":"0604d7c271304a5c9ed9fd4942bfe482be5dfc16666c47a0c9e7ed0fcce2418e"} Jul 15 17:00:06.508660 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.508606 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bbtx8" event={"ID":"5ba4bd8c-b016-4515-865b-6977c37af13e","Type":"ContainerStarted","Data":"d4b9bd283e2315512b9725f7f96da15eed6b749b015f8c3fe9e658d9151814c8"} Jul 15 17:00:06.511482 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.511445 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-td2hl" event={"ID":"c86c79e5-143a-4be2-b192-8192c6dedc20","Type":"ContainerStarted","Data":"21934ac30b5cfa47d6da32fc7f04d70a0db2cbde117dfac6a3bb08a783e4b4de"} Jul 15 17:00:06.512999 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.512899 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36","Type":"ContainerStarted","Data":"7a967739f6ce400a069cd82306e747045bed4485d2000052b5abca768c58ead6"} Jul 15 17:00:06.514909 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.514801 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" event={"ID":"8506444a-b35c-474f-84e0-e3bceb00271a","Type":"ContainerStarted","Data":"c34118621b3658ae59a87891cf92138c5152ec41f0d855e6929d67fba37b880e"} Jul 15 17:00:06.518194 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.517913 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-td2hl" Jul 15 17:00:06.519714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.519652 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvmkg" event={"ID":"7e14736e-6d94-4699-89e4-3a05998d1afe","Type":"ContainerStarted","Data":"05c69d78df2c3f437f9d37557e7cde79007cc8eed0932c718dab89e3db89104a"} Jul 15 17:00:06.520328 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.520307 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-td2hl" Jul 15 17:00:06.538553 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.538221 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6r8xd" podStartSLOduration=17.728304753 podStartE2EDuration="30.538199813s" totalImagesPullingTime="12.80989506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:06.537253918 +0000 UTC m=+56.779249150" watchObservedRunningTime="2026-07-15 17:00:06.538199813 +0000 UTC m=+56.780195043" Jul 15 17:00:06.556004 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.555818 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r92qp" podStartSLOduration=17.084418235 podStartE2EDuration="29.555801055s" totalImagesPullingTime="12.47138282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:06.555621068 +0000 UTC m=+56.797616295" watchObservedRunningTime="2026-07-15 17:00:06.555801055 +0000 UTC m=+56.797796282" Jul 15 17:00:06.597694 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.597635 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rjhcq" podStartSLOduration=1.565696542 podStartE2EDuration="14.597616209s" totalImagesPullingTime="13.031919667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:06.577033187 +0000 UTC m=+56.819028413" watchObservedRunningTime="2026-07-15 17:00:06.597616209 +0000 UTC m=+56.839611428" Jul 15 17:00:06.598618 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.598577 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-td2hl" podStartSLOduration=11.336579786 podStartE2EDuration="14.598562687s" totalImagesPullingTime="3.261982901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:06.59698617 +0000 UTC m=+56.838981399" watchObservedRunningTime="2026-07-15 17:00:06.598562687 +0000 UTC m=+56.840557914" Jul 15 17:00:06.657516 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:06.657393 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c4b99847c-2vf5v"] Jul 15 17:00:06.662893 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:00:06.662818 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ffce7c9_7583_4e7a_a194_8b0827bc2ee1.slice/crio-30c7a81ab133eaf07913cbf43a8993fc06dbf103d4ce02e44f8fe2ef5c853542 WatchSource:0}: Error finding container 30c7a81ab133eaf07913cbf43a8993fc06dbf103d4ce02e44f8fe2ef5c853542: Status 404 returned error can't find the container with id 30c7a81ab133eaf07913cbf43a8993fc06dbf103d4ce02e44f8fe2ef5c853542 Jul 15 17:00:07.102157 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:07.102065 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:07.528598 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:07.528515 2562 generic.go:356] "Generic (PLEG): container finished" podID="35c01559-3824-4e5c-bfea-71ead02ea8ed" containerID="4bb0237914130d5fcd46c7cf581f3cb0206d0c38c56d6cb14f9952b3c5288f95" exitCode=0 Jul 15 17:00:07.528598 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:07.528561 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxs9l" event={"ID":"35c01559-3824-4e5c-bfea-71ead02ea8ed","Type":"ContainerDied","Data":"4bb0237914130d5fcd46c7cf581f3cb0206d0c38c56d6cb14f9952b3c5288f95"} Jul 15 17:00:07.531324 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:07.531293 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" event={"ID":"7adfd351-5cdc-43c3-8684-553480b8ab6f","Type":"ContainerStarted","Data":"03ddba82c17333a0b7e7b12479de2f7eaedc5f937bdfeeab804e5b5d1bfe9e81"} Jul 15 17:00:07.533164 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:07.533136 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" event={"ID":"9a7be7b5-4e96-4075-9880-ed9aed255eab","Type":"ContainerStarted","Data":"690cc4194667d3db677ff92bfa2dfcccc4b8c66ab6ab54c64053d762989d185f"} Jul 15 17:00:07.534797 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:07.534765 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" event={"ID":"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1","Type":"ContainerStarted","Data":"30c7a81ab133eaf07913cbf43a8993fc06dbf103d4ce02e44f8fe2ef5c853542"} Jul 15 17:00:08.541007 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:08.540976 2562 generic.go:356] "Generic (PLEG): container finished" podID="7e14736e-6d94-4699-89e4-3a05998d1afe" containerID="183e93cddf1782cf554b5888464e4c771d85e956eeb74a4df4a4bcbf5270b908" exitCode=0 Jul 15 17:00:08.541335 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:08.541189 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvmkg" event={"ID":"7e14736e-6d94-4699-89e4-3a05998d1afe","Type":"ContainerDied","Data":"183e93cddf1782cf554b5888464e4c771d85e956eeb74a4df4a4bcbf5270b908"} Jul 15 17:00:08.547429 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:08.547400 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxs9l" event={"ID":"35c01559-3824-4e5c-bfea-71ead02ea8ed","Type":"ContainerStarted","Data":"00a5a750854571e5fcfef45352375679369955cddcb9f5d1f4f16a5545f5f979"} Jul 15 17:00:08.607243 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:08.606061 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cxs9l" podStartSLOduration=5.243889659 podStartE2EDuration="31.606041415s" totalImagesPullingTime="26.362151756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:08.604217564 +0000 UTC m=+58.846212790" watchObservedRunningTime="2026-07-15 17:00:08.606041415 +0000 UTC m=+58.848036642" Jul 15 17:00:08.759572 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:08.759541 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:08.936829 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:08.936779 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:09.556972 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:09.556883 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" event={"ID":"8506444a-b35c-474f-84e0-e3bceb00271a","Type":"ContainerStarted","Data":"37774a2bee260e5a4302b32a5e586973da3f8d9d103549aa4ef5e3ec881de253"} Jul 15 17:00:09.557404 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:09.556987 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" event={"ID":"8506444a-b35c-474f-84e0-e3bceb00271a","Type":"ContainerStarted","Data":"6aa5a37765f7147d45973de173041bed63fc82e68497370916ab8c3b367788f2"} Jul 15 17:00:09.560285 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:09.560253 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvmkg" event={"ID":"7e14736e-6d94-4699-89e4-3a05998d1afe","Type":"ContainerStarted","Data":"f2faa2235ca669000a6a207e43b878d01075ea772b5b6a8321fc66852ade6d26"} Jul 15 17:00:09.560424 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:09.560305 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvmkg" event={"ID":"7e14736e-6d94-4699-89e4-3a05998d1afe","Type":"ContainerStarted","Data":"f7fec05066a888ade14d87beaced40dae73609072c4e8c0cfc063843aad87dea"} Jul 15 17:00:09.600684 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:09.600624 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vvmkg" podStartSLOduration=7.124112622 podStartE2EDuration="9.60060288s" totalImagesPullingTime="2.372535643s" totalInitContainerRuntime="103.954615ms" isStatefulPod=true podCreationTimestamp="2026-07-15 17:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:09.600069638 +0000 UTC m=+59.842064865" watchObservedRunningTime="2026-07-15 17:00:09.60060288 +0000 UTC m=+59.842598126" Jul 15 17:00:11.057232 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:11.057191 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:11.400786 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:11.400745 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:11.672651 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:11.672532 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:11.869682 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:11.869606 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:12.037056 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:12.036960 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:12.905145 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:12.905099 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:13.039080 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.039052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:13.583287 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.583243 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bbtx8" event={"ID":"5ba4bd8c-b016-4515-865b-6977c37af13e","Type":"ContainerStarted","Data":"53d6b91efe86b6fc42a15ba8915e72e0b6b894c7544e2f18f00840d511670b37"} Jul 15 17:00:13.585236 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.585206 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" event={"ID":"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1","Type":"ContainerStarted","Data":"642a8d9ca37d13e24baac7cbab2e7a3968fd201a320917200cdfa3939c0b03f5"} Jul 15 17:00:13.585441 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.585265 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" event={"ID":"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1","Type":"ContainerStarted","Data":"22a6a8ce54fbf74471bec35e9dfc58ca55cfb7ae5747697f3e52fc5ec6d27033"} Jul 15 17:00:13.585441 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.585292 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" event={"ID":"7ffce7c9-7583-4e7a-a194-8b0827bc2ee1","Type":"ContainerStarted","Data":"ad03ac7f54150ed1a1b305f4eeedefbb460e6281fdc352d03bba1ad83d46a06e"} Jul 15 17:00:13.586631 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.586609 2562 generic.go:356] "Generic (PLEG): container finished" podID="87a18837-b8d6-47ce-89d7-bc3fbfc1cc36" containerID="af788e0ee065f136879c3f60f0756672009c8cf5fd775304d5ddd98bfabc2be5" exitCode=0 Jul 15 17:00:13.586737 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.586692 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36","Type":"ContainerDied","Data":"af788e0ee065f136879c3f60f0756672009c8cf5fd775304d5ddd98bfabc2be5"} Jul 15 17:00:13.588891 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.588872 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" event={"ID":"8506444a-b35c-474f-84e0-e3bceb00271a","Type":"ContainerStarted","Data":"0ece688b37589835e7e347a0e910118d3059b9caaa72fc693c5fc4f5c30ac6bc"} Jul 15 17:00:13.591147 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.591115 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" event={"ID":"1553e479-bbd5-40d4-b7b1-060572599bb9","Type":"ContainerStarted","Data":"b84186ee40a645531434321f3ed30c1efc0880d1c96082f89bc149d6ab441535"} Jul 15 17:00:13.591238 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.591172 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" event={"ID":"1553e479-bbd5-40d4-b7b1-060572599bb9","Type":"ContainerStarted","Data":"93d06e098a29ce6ee5dbc35f8fd0605021fbf92019b22ac80ad1ef43c5ebf683"} Jul 15 17:00:13.591238 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.591207 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" event={"ID":"1553e479-bbd5-40d4-b7b1-060572599bb9","Type":"ContainerStarted","Data":"d7fad2fb4d6ce1f0cdbc53a6f37138790f1d3c8bab44d3cf9562c1ca3fb3e277"} Jul 15 17:00:13.593315 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.593292 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" event={"ID":"7adfd351-5cdc-43c3-8684-553480b8ab6f","Type":"ContainerStarted","Data":"b0206a79f0660b24ca897726d3a15f63fd95c0b11f064e2904bef978792874d0"} Jul 15 17:00:13.594696 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.594677 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" event={"ID":"9a7be7b5-4e96-4075-9880-ed9aed255eab","Type":"ContainerStarted","Data":"f74d2045ed0db621c145e677834596f29df618a73f45f9e19de8deff18d24546"} Jul 15 17:00:13.594894 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.594873 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" Jul 15 17:00:13.599600 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.599583 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" Jul 15 17:00:13.624255 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.624200 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6c4b99847c-2vf5v" podStartSLOduration=1.10492186 podStartE2EDuration="7.624184343s" totalImagesPullingTime="6.519262483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:13.619706059 +0000 UTC m=+63.861701284" watchObservedRunningTime="2026-07-15 17:00:13.624184343 +0000 UTC m=+63.866179569" Jul 15 17:00:13.650034 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.649995 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-lpq5l" podStartSLOduration=11.475824629 podStartE2EDuration="13.64998292s" totalImagesPullingTime="2.174158291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:13.649379134 +0000 UTC m=+63.891374359" watchObservedRunningTime="2026-07-15 17:00:13.64998292 +0000 UTC m=+63.891978123" Jul 15 17:00:13.770378 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.770296 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-rctbg" podStartSLOduration=2.063849868 podStartE2EDuration="8.77028152s" totalImagesPullingTime="6.706431652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:13.76816908 +0000 UTC m=+64.010164306" watchObservedRunningTime="2026-07-15 17:00:13.77028152 +0000 UTC m=+64.012276746" Jul 15 17:00:13.846857 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:13.846792 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-npdxr" podStartSLOduration=7.941632778 podStartE2EDuration="13.846775877s" totalImagesPullingTime="5.905143099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:13.846520372 +0000 UTC m=+64.088515599" watchObservedRunningTime="2026-07-15 17:00:13.846775877 +0000 UTC m=+64.088771146" Jul 15 17:00:15.607604 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:15.607569 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" event={"ID":"1553e479-bbd5-40d4-b7b1-060572599bb9","Type":"ContainerStarted","Data":"ebfa23185a9adfc83ac542e2496986beff5f3a2853c4bfca88678d5ea36b8b36"} Jul 15 17:00:15.607985 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:15.607647 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" event={"ID":"1553e479-bbd5-40d4-b7b1-060572599bb9","Type":"ContainerStarted","Data":"3c55b0c8ea687b3d23ae2bc38312cf4dc6784c47a87eedafbda49789a27bb3ac"} Jul 15 17:00:15.607985 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:15.607689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" event={"ID":"1553e479-bbd5-40d4-b7b1-060572599bb9","Type":"ContainerStarted","Data":"e52966938c0d435c62ac56b25312eacc4cca6def0f0b25b93751e8922956abaa"} Jul 15 17:00:15.645937 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:15.645882 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" podStartSLOduration=4.519210579 podStartE2EDuration="12.645863038s" totalImagesPullingTime="8.126652459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:15.642874651 +0000 UTC m=+65.884869878" watchObservedRunningTime="2026-07-15 17:00:15.645863038 +0000 UTC m=+65.887858264" Jul 15 17:00:16.613507 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:16.613469 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:18.621990 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:18.621915 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bbtx8" event={"ID":"5ba4bd8c-b016-4515-865b-6977c37af13e","Type":"ContainerStarted","Data":"a4bb3939197d05b12758096f8a15a7c2a53d1226cc32def6f9d9712e9f5151e5"} Jul 15 17:00:18.625098 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:18.625076 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36","Type":"ContainerStarted","Data":"e58b5b06ebb51bcd7b4f5c9fc4bf82942538bc3640f0a8f49469b70880cdb058"} Jul 15 17:00:18.625216 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:18.625140 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36","Type":"ContainerStarted","Data":"0af624e91f7ec576a0cb6e69f0b6143050291dbf6e391fe3fa485a1f0f50d2df"} Jul 15 17:00:18.625216 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:18.625180 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36","Type":"ContainerStarted","Data":"2b1bad8a7c3d446b36b0335f9f08461695c897a6c8387253be6787238ad3d49f"} Jul 15 17:00:18.625332 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:18.625218 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36","Type":"ContainerStarted","Data":"38c8e6c866effdcb37300ed3cf4922ceebd42de910f90edfe50af5842b66f67c"} Jul 15 17:00:18.625332 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:18.625258 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36","Type":"ContainerStarted","Data":"4a89b59fd5c3d2a41f3817a94ec4a317d89cc474ff2711913203ad238b403e2a"} Jul 15 17:00:18.625332 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:18.625297 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"87a18837-b8d6-47ce-89d7-bc3fbfc1cc36","Type":"ContainerStarted","Data":"f866e23b173e188d328007047bf01295cd523c0d48d7bbdab9225559cd24a8b8"} Jul 15 17:00:18.656412 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:18.656365 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bbtx8" podStartSLOduration=1.820687199 podStartE2EDuration="26.65634736s" totalImagesPullingTime="24.835660161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:18.654324379 +0000 UTC m=+68.896319616" watchObservedRunningTime="2026-07-15 17:00:18.65634736 +0000 UTC m=+68.898342590" Jul 15 17:00:18.707177 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:18.707130 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.560766829 podStartE2EDuration="17.707115237s" totalImagesPullingTime="11.113186006s" totalInitContainerRuntime="33.162402ms" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:18.705817069 +0000 UTC m=+68.947812294" watchObservedRunningTime="2026-07-15 17:00:18.707115237 +0000 UTC m=+68.949110464" Jul 15 17:00:19.872362 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:19.872315 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:19.986113 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:19.986073 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:22.622528 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:22.622500 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-749db7cfc4-tw8m9" Jul 15 17:00:26.403602 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:26.403572 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:26.482123 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:26.482092 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:26.834431 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:26.834394 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:26.934919 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:26.934890 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:00:37.537722 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:37.537598 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r92qp" Jul 15 17:00:52.621801 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:00:52.621762 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:04.237112 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:04.237072 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:04.599238 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:04.599205 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:16.875911 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:16.875869 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:16.930030 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:16.930003 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:28.763071 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:28.763032 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:28.828511 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:28.828474 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:30.965029 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:30.964997 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:41.870372 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:41.870332 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:41.968494 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:41.968457 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:42.228122 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:42.228046 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:42.330020 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:42.329986 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:42.400613 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:42.400580 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:46.527755 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:46.527719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:49.307623 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:49.307566 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:01:49.373038 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:01:49.373004 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:30.838460 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:30.838418 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:30.938035 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:30.938005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:31.434548 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.434518 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk"] Jul 15 17:02:31.437865 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.437828 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:31.440766 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.440743 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xjkzd\"" Jul 15 17:02:31.440897 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.440799 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 17:02:31.440897 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.440807 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 17:02:31.448431 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.448409 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk"] Jul 15 17:02:31.526699 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.526669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnwvp\" (UniqueName: \"kubernetes.io/projected/1617aa8e-d990-439e-bd50-2be47e0aa30d-kube-api-access-bnwvp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:31.526815 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.526715 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:31.526815 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.526749 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:31.627530 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.627502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnwvp\" (UniqueName: \"kubernetes.io/projected/1617aa8e-d990-439e-bd50-2be47e0aa30d-kube-api-access-bnwvp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:31.627661 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.627547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:31.627661 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.627576 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:31.627991 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.627960 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:31.628103 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.627972 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:31.636422 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.636395 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnwvp\" (UniqueName: \"kubernetes.io/projected/1617aa8e-d990-439e-bd50-2be47e0aa30d-kube-api-access-bnwvp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:31.747219 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.747146 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:31.876344 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:31.876320 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk"] Jul 15 17:02:32.138083 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:32.138043 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" event={"ID":"1617aa8e-d990-439e-bd50-2be47e0aa30d","Type":"ContainerStarted","Data":"d2e8786b50670ccc5cb81da1e156c881bad3824894fb0f0f170d4a8392d88ea3"} Jul 15 17:02:42.178958 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:42.178871 2562 generic.go:356] "Generic (PLEG): container finished" podID="1617aa8e-d990-439e-bd50-2be47e0aa30d" containerID="b9cd7dc889f2ddb333c74dd75c3067711043e774fea9a02d68ef2940cfeb69cd" exitCode=0 Jul 15 17:02:42.178958 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:42.178933 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" event={"ID":"1617aa8e-d990-439e-bd50-2be47e0aa30d","Type":"ContainerDied","Data":"b9cd7dc889f2ddb333c74dd75c3067711043e774fea9a02d68ef2940cfeb69cd"} Jul 15 17:02:45.192287 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:45.192248 2562 generic.go:356] "Generic (PLEG): container finished" podID="1617aa8e-d990-439e-bd50-2be47e0aa30d" containerID="473fb10653b9b23cf03a4d1a87c5758adf90aacff14978072fd1821819f32572" exitCode=0 Jul 15 17:02:45.192287 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:45.192279 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" event={"ID":"1617aa8e-d990-439e-bd50-2be47e0aa30d","Type":"ContainerDied","Data":"473fb10653b9b23cf03a4d1a87c5758adf90aacff14978072fd1821819f32572"} Jul 15 17:02:55.233542 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:55.233506 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" event={"ID":"1617aa8e-d990-439e-bd50-2be47e0aa30d","Type":"ContainerStarted","Data":"7db36ed93dedf56f7a6e792f7a58ad0b1555f7146086b78a30ce9750bedb8186"} Jul 15 17:02:55.258730 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:55.258679 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" podStartSLOduration=2.3249043130000002 podStartE2EDuration="24.258661802s" totalImagesPullingTime="21.933757489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:02:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:02:55.257109511 +0000 UTC m=+225.499104761" watchObservedRunningTime="2026-07-15 17:02:55.258661802 +0000 UTC m=+225.500657030" Jul 15 17:02:56.239561 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:56.239525 2562 generic.go:356] "Generic (PLEG): container finished" podID="1617aa8e-d990-439e-bd50-2be47e0aa30d" containerID="7db36ed93dedf56f7a6e792f7a58ad0b1555f7146086b78a30ce9750bedb8186" exitCode=0 Jul 15 17:02:56.239979 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:56.239582 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" event={"ID":"1617aa8e-d990-439e-bd50-2be47e0aa30d","Type":"ContainerDied","Data":"7db36ed93dedf56f7a6e792f7a58ad0b1555f7146086b78a30ce9750bedb8186"} Jul 15 17:02:56.598673 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:56.598632 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:57.368022 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:57.367999 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:57.453862 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:57.453807 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-util\") pod \"1617aa8e-d990-439e-bd50-2be47e0aa30d\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " Jul 15 17:02:57.454026 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:57.453911 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnwvp\" (UniqueName: \"kubernetes.io/projected/1617aa8e-d990-439e-bd50-2be47e0aa30d-kube-api-access-bnwvp\") pod \"1617aa8e-d990-439e-bd50-2be47e0aa30d\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " Jul 15 17:02:57.454026 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:57.453938 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-bundle\") pod \"1617aa8e-d990-439e-bd50-2be47e0aa30d\" (UID: \"1617aa8e-d990-439e-bd50-2be47e0aa30d\") " Jul 15 17:02:57.454574 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:57.454544 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-bundle" (OuterVolumeSpecName: "bundle") pod "1617aa8e-d990-439e-bd50-2be47e0aa30d" (UID: "1617aa8e-d990-439e-bd50-2be47e0aa30d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:02:57.456215 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:57.456180 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1617aa8e-d990-439e-bd50-2be47e0aa30d-kube-api-access-bnwvp" (OuterVolumeSpecName: "kube-api-access-bnwvp") pod "1617aa8e-d990-439e-bd50-2be47e0aa30d" (UID: "1617aa8e-d990-439e-bd50-2be47e0aa30d"). InnerVolumeSpecName "kube-api-access-bnwvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:02:57.458768 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:57.458716 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-util" (OuterVolumeSpecName: "util") pod "1617aa8e-d990-439e-bd50-2be47e0aa30d" (UID: "1617aa8e-d990-439e-bd50-2be47e0aa30d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:02:57.555138 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:57.555063 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-util\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:02:57.555138 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:57.555087 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnwvp\" (UniqueName: \"kubernetes.io/projected/1617aa8e-d990-439e-bd50-2be47e0aa30d-kube-api-access-bnwvp\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:02:57.555138 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:57.555097 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1617aa8e-d990-439e-bd50-2be47e0aa30d-bundle\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:02:58.248623 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:58.248596 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" Jul 15 17:02:58.250643 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:58.250609 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xv9bk" event={"ID":"1617aa8e-d990-439e-bd50-2be47e0aa30d","Type":"ContainerDied","Data":"d2e8786b50670ccc5cb81da1e156c881bad3824894fb0f0f170d4a8392d88ea3"} Jul 15 17:02:58.250762 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:02:58.250664 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2e8786b50670ccc5cb81da1e156c881bad3824894fb0f0f170d4a8392d88ea3" Jul 15 17:03:02.538426 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:02.538391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:03.421709 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:03.421665 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:04.456407 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.456377 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth"] Jul 15 17:03:04.456791 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.456660 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1617aa8e-d990-439e-bd50-2be47e0aa30d" containerName="pull" Jul 15 17:03:04.456791 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.456671 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1617aa8e-d990-439e-bd50-2be47e0aa30d" containerName="pull" Jul 15 17:03:04.456791 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.456678 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1617aa8e-d990-439e-bd50-2be47e0aa30d" containerName="extract" Jul 15 17:03:04.456791 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.456683 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1617aa8e-d990-439e-bd50-2be47e0aa30d" containerName="extract" Jul 15 17:03:04.456791 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.456705 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1617aa8e-d990-439e-bd50-2be47e0aa30d" containerName="util" Jul 15 17:03:04.456791 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.456711 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1617aa8e-d990-439e-bd50-2be47e0aa30d" containerName="util" Jul 15 17:03:04.456791 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.456751 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1617aa8e-d990-439e-bd50-2be47e0aa30d" containerName="extract" Jul 15 17:03:04.459775 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.459759 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth" Jul 15 17:03:04.463620 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.463599 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 15 17:03:04.463771 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.463659 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-m7z4c\"" Jul 15 17:03:04.463937 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.463922 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 15 17:03:04.481058 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.481034 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth"] Jul 15 17:03:04.506963 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.506931 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttpmq\" (UniqueName: \"kubernetes.io/projected/4aa7c6d8-a605-411d-a965-90980e1ea554-kube-api-access-ttpmq\") pod \"cert-manager-operator-controller-manager-55c444bf86-jgfth\" (UID: \"4aa7c6d8-a605-411d-a965-90980e1ea554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth" Jul 15 17:03:04.507111 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.506975 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4aa7c6d8-a605-411d-a965-90980e1ea554-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jgfth\" (UID: \"4aa7c6d8-a605-411d-a965-90980e1ea554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth" Jul 15 17:03:04.607606 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.607565 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttpmq\" (UniqueName: \"kubernetes.io/projected/4aa7c6d8-a605-411d-a965-90980e1ea554-kube-api-access-ttpmq\") pod \"cert-manager-operator-controller-manager-55c444bf86-jgfth\" (UID: \"4aa7c6d8-a605-411d-a965-90980e1ea554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth" Jul 15 17:03:04.607606 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.607614 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4aa7c6d8-a605-411d-a965-90980e1ea554-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jgfth\" (UID: \"4aa7c6d8-a605-411d-a965-90980e1ea554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth" Jul 15 17:03:04.608016 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.607997 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4aa7c6d8-a605-411d-a965-90980e1ea554-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jgfth\" (UID: \"4aa7c6d8-a605-411d-a965-90980e1ea554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth" Jul 15 17:03:04.617717 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.617699 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttpmq\" (UniqueName: \"kubernetes.io/projected/4aa7c6d8-a605-411d-a965-90980e1ea554-kube-api-access-ttpmq\") pod \"cert-manager-operator-controller-manager-55c444bf86-jgfth\" (UID: \"4aa7c6d8-a605-411d-a965-90980e1ea554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth" Jul 15 17:03:04.768695 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.768615 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth" Jul 15 17:03:04.905710 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:04.905679 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth"] Jul 15 17:03:04.909151 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:03:04.909120 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa7c6d8_a605_411d_a965_90980e1ea554.slice/crio-558fe6c8931084419d16c0a0a5541f2c878035f654ee0e64ba6a46fd9d8b9a63 WatchSource:0}: Error finding container 558fe6c8931084419d16c0a0a5541f2c878035f654ee0e64ba6a46fd9d8b9a63: Status 404 returned error can't find the container with id 558fe6c8931084419d16c0a0a5541f2c878035f654ee0e64ba6a46fd9d8b9a63 Jul 15 17:03:05.275158 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:05.275120 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth" event={"ID":"4aa7c6d8-a605-411d-a965-90980e1ea554","Type":"ContainerStarted","Data":"558fe6c8931084419d16c0a0a5541f2c878035f654ee0e64ba6a46fd9d8b9a63"} Jul 15 17:03:06.141771 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:06.141736 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:06.212726 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:06.212690 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:07.284571 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:07.284532 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth" event={"ID":"4aa7c6d8-a605-411d-a965-90980e1ea554","Type":"ContainerStarted","Data":"05c526c738bdeb8ae15ac1df4ce2d64648e02ca1e193b8c43cc23f86e0064ed2"} Jul 15 17:03:07.307305 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:07.307147 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jgfth" podStartSLOduration=1.50875606 podStartE2EDuration="3.307121784s" totalImagesPullingTime="1.798365724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:03:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:03:07.305870247 +0000 UTC m=+237.547865473" watchObservedRunningTime="2026-07-15 17:03:07.307121784 +0000 UTC m=+237.549117012" Jul 15 17:03:07.569189 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:07.569114 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:07.693954 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:07.693919 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:07.851428 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:07.851395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:08.333709 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.333673 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:08.727872 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.727780 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7q999"] Jul 15 17:03:08.731058 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.731038 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" Jul 15 17:03:08.734118 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.734096 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 17:03:08.735229 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.735213 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 17:03:08.735324 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.735219 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-mk5k9\"" Jul 15 17:03:08.746310 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.746288 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7q999"] Jul 15 17:03:08.838852 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.838801 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:08.843786 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.843767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljzm7\" (UniqueName: \"kubernetes.io/projected/beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c-kube-api-access-ljzm7\") pod \"cert-manager-webhook-6486ffd997-7q999\" (UID: \"beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" Jul 15 17:03:08.843873 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.843809 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7q999\" (UID: \"beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" Jul 15 17:03:08.944211 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.944183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7q999\" (UID: \"beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" Jul 15 17:03:08.944339 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.944239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljzm7\" (UniqueName: \"kubernetes.io/projected/beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c-kube-api-access-ljzm7\") pod \"cert-manager-webhook-6486ffd997-7q999\" (UID: \"beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" Jul 15 17:03:08.953577 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.953551 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7q999\" (UID: \"beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" Jul 15 17:03:08.954126 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.954104 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljzm7\" (UniqueName: \"kubernetes.io/projected/beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c-kube-api-access-ljzm7\") pod \"cert-manager-webhook-6486ffd997-7q999\" (UID: \"beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" Jul 15 17:03:08.985336 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.985276 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj"] Jul 15 17:03:08.988723 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.988707 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:08.991490 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.991474 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 17:03:08.991853 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.991822 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 17:03:08.992093 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.992080 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xjkzd\"" Jul 15 17:03:08.999168 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:08.999151 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj"] Jul 15 17:03:09.045536 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.045509 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cmtc\" (UniqueName: \"kubernetes.io/projected/d496b4c6-065b-440a-86aa-6245e2ed14cf-kube-api-access-4cmtc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:09.045631 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.045578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:09.045631 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.045606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:09.052500 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.052477 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" Jul 15 17:03:09.146336 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.146307 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:09.146474 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.146346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:09.146531 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.146485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cmtc\" (UniqueName: \"kubernetes.io/projected/d496b4c6-065b-440a-86aa-6245e2ed14cf-kube-api-access-4cmtc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:09.146727 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.146708 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:09.146781 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.146728 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:09.155371 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.155348 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cmtc\" (UniqueName: \"kubernetes.io/projected/d496b4c6-065b-440a-86aa-6245e2ed14cf-kube-api-access-4cmtc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:09.181286 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.181265 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7q999"] Jul 15 17:03:09.183104 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:03:09.183083 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbeb0ffd6_9f3d_4d4b_9ed2_cbefb5e7269c.slice/crio-f1506e2fddc132e7d377b9f5a5b5cba7de2416ceafd4641caac116fe6214b326 WatchSource:0}: Error finding container f1506e2fddc132e7d377b9f5a5b5cba7de2416ceafd4641caac116fe6214b326: Status 404 returned error can't find the container with id f1506e2fddc132e7d377b9f5a5b5cba7de2416ceafd4641caac116fe6214b326 Jul 15 17:03:09.293062 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.293001 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" event={"ID":"beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c","Type":"ContainerStarted","Data":"f1506e2fddc132e7d377b9f5a5b5cba7de2416ceafd4641caac116fe6214b326"} Jul 15 17:03:09.298161 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.298143 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:09.313948 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.313914 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:09.413279 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.413248 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:09.431027 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.431008 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj"] Jul 15 17:03:09.432891 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:03:09.432861 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd496b4c6_065b_440a_86aa_6245e2ed14cf.slice/crio-aa462fcfce66027447edfb8742ed1f60f36b2d39f3e1c5b651d2a548f70050cf WatchSource:0}: Error finding container aa462fcfce66027447edfb8742ed1f60f36b2d39f3e1c5b651d2a548f70050cf: Status 404 returned error can't find the container with id aa462fcfce66027447edfb8742ed1f60f36b2d39f3e1c5b651d2a548f70050cf Jul 15 17:03:09.964929 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:09.964881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:10.297523 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:10.297447 2562 generic.go:356] "Generic (PLEG): container finished" podID="d496b4c6-065b-440a-86aa-6245e2ed14cf" containerID="39bcf3761ea21d92f279292dad931c5ad8e6917aca05fe7bf9ffd9934b94b06e" exitCode=0 Jul 15 17:03:10.297665 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:10.297525 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" event={"ID":"d496b4c6-065b-440a-86aa-6245e2ed14cf","Type":"ContainerDied","Data":"39bcf3761ea21d92f279292dad931c5ad8e6917aca05fe7bf9ffd9934b94b06e"} Jul 15 17:03:10.297665 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:10.297582 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" event={"ID":"d496b4c6-065b-440a-86aa-6245e2ed14cf","Type":"ContainerStarted","Data":"aa462fcfce66027447edfb8742ed1f60f36b2d39f3e1c5b651d2a548f70050cf"} Jul 15 17:03:12.308479 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:12.308447 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" event={"ID":"d496b4c6-065b-440a-86aa-6245e2ed14cf","Type":"ContainerStarted","Data":"1cf55a156449a286fc1c386b7d265adf75b6f7d58037ab3ad1043a5ca10645aa"} Jul 15 17:03:12.949372 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:12.949336 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:13.049903 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:13.049871 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:13.315211 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:13.315127 2562 generic.go:356] "Generic (PLEG): container finished" podID="d496b4c6-065b-440a-86aa-6245e2ed14cf" containerID="1cf55a156449a286fc1c386b7d265adf75b6f7d58037ab3ad1043a5ca10645aa" exitCode=0 Jul 15 17:03:13.315652 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:13.315223 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" event={"ID":"d496b4c6-065b-440a-86aa-6245e2ed14cf","Type":"ContainerDied","Data":"1cf55a156449a286fc1c386b7d265adf75b6f7d58037ab3ad1043a5ca10645aa"} Jul 15 17:03:14.322379 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:14.322350 2562 generic.go:356] "Generic (PLEG): container finished" podID="d496b4c6-065b-440a-86aa-6245e2ed14cf" containerID="49624d3c3c7155bb86ef1bd85440d5b6663d67bcb4a9bc61c4d7e7d60deb9f7a" exitCode=0 Jul 15 17:03:14.322828 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:14.322390 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" event={"ID":"d496b4c6-065b-440a-86aa-6245e2ed14cf","Type":"ContainerDied","Data":"49624d3c3c7155bb86ef1bd85440d5b6663d67bcb4a9bc61c4d7e7d60deb9f7a"} Jul 15 17:03:15.328449 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.328403 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" event={"ID":"beb0ffd6-9f3d-4d4b-9ed2-cbefb5e7269c","Type":"ContainerStarted","Data":"8292b69083ebc1715008c6400e20026cca4ba43b4508f6bf60c070bd9c1b4d64"} Jul 15 17:03:15.328913 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.328646 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" Jul 15 17:03:15.367215 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.367170 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" podStartSLOduration=1.995525175 podStartE2EDuration="7.367153675s" totalImagesPullingTime="5.3716285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:03:15.366878569 +0000 UTC m=+245.608873796" watchObservedRunningTime="2026-07-15 17:03:15.367153675 +0000 UTC m=+245.609148900" Jul 15 17:03:15.452207 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.452182 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:15.505281 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.505254 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-util\") pod \"d496b4c6-065b-440a-86aa-6245e2ed14cf\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " Jul 15 17:03:15.505423 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.505323 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-bundle\") pod \"d496b4c6-065b-440a-86aa-6245e2ed14cf\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " Jul 15 17:03:15.505423 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.505359 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cmtc\" (UniqueName: \"kubernetes.io/projected/d496b4c6-065b-440a-86aa-6245e2ed14cf-kube-api-access-4cmtc\") pod \"d496b4c6-065b-440a-86aa-6245e2ed14cf\" (UID: \"d496b4c6-065b-440a-86aa-6245e2ed14cf\") " Jul 15 17:03:15.505708 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.505682 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-bundle" (OuterVolumeSpecName: "bundle") pod "d496b4c6-065b-440a-86aa-6245e2ed14cf" (UID: "d496b4c6-065b-440a-86aa-6245e2ed14cf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:03:15.507585 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.507559 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d496b4c6-065b-440a-86aa-6245e2ed14cf-kube-api-access-4cmtc" (OuterVolumeSpecName: "kube-api-access-4cmtc") pod "d496b4c6-065b-440a-86aa-6245e2ed14cf" (UID: "d496b4c6-065b-440a-86aa-6245e2ed14cf"). InnerVolumeSpecName "kube-api-access-4cmtc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:03:15.509530 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.509510 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-util" (OuterVolumeSpecName: "util") pod "d496b4c6-065b-440a-86aa-6245e2ed14cf" (UID: "d496b4c6-065b-440a-86aa-6245e2ed14cf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:03:15.606289 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.606207 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-bundle\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:03:15.606289 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.606243 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cmtc\" (UniqueName: \"kubernetes.io/projected/d496b4c6-065b-440a-86aa-6245e2ed14cf-kube-api-access-4cmtc\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:03:15.606289 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:15.606253 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d496b4c6-065b-440a-86aa-6245e2ed14cf-util\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:03:16.333596 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:16.333571 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" Jul 15 17:03:16.333596 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:16.333580 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhwdxj" event={"ID":"d496b4c6-065b-440a-86aa-6245e2ed14cf","Type":"ContainerDied","Data":"aa462fcfce66027447edfb8742ed1f60f36b2d39f3e1c5b651d2a548f70050cf"} Jul 15 17:03:16.334012 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:16.333630 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa462fcfce66027447edfb8742ed1f60f36b2d39f3e1c5b651d2a548f70050cf" Jul 15 17:03:18.757900 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:18.757861 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:19.315572 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.315540 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-v7znw"] Jul 15 17:03:19.315848 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.315821 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d496b4c6-065b-440a-86aa-6245e2ed14cf" containerName="util" Jul 15 17:03:19.315896 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.315850 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d496b4c6-065b-440a-86aa-6245e2ed14cf" containerName="util" Jul 15 17:03:19.315896 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.315869 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d496b4c6-065b-440a-86aa-6245e2ed14cf" containerName="pull" Jul 15 17:03:19.315896 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.315874 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d496b4c6-065b-440a-86aa-6245e2ed14cf" containerName="pull" Jul 15 17:03:19.315896 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.315883 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d496b4c6-065b-440a-86aa-6245e2ed14cf" containerName="extract" Jul 15 17:03:19.315896 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.315889 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d496b4c6-065b-440a-86aa-6245e2ed14cf" containerName="extract" Jul 15 17:03:19.316047 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.315935 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d496b4c6-065b-440a-86aa-6245e2ed14cf" containerName="extract" Jul 15 17:03:19.319095 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.319078 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7znw" Jul 15 17:03:19.321566 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.321545 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fh8pp\"" Jul 15 17:03:19.329865 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.329817 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-v7znw"] Jul 15 17:03:19.432940 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.432914 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0217870b-d6c5-4085-b929-1d81c3c860ce-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-v7znw\" (UID: \"0217870b-d6c5-4085-b929-1d81c3c860ce\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7znw" Jul 15 17:03:19.433064 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.432970 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdmnv\" (UniqueName: \"kubernetes.io/projected/0217870b-d6c5-4085-b929-1d81c3c860ce-kube-api-access-qdmnv\") pod \"cert-manager-cainjector-559cdd8c87-v7znw\" (UID: \"0217870b-d6c5-4085-b929-1d81c3c860ce\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7znw" Jul 15 17:03:19.534025 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.533999 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdmnv\" (UniqueName: \"kubernetes.io/projected/0217870b-d6c5-4085-b929-1d81c3c860ce-kube-api-access-qdmnv\") pod \"cert-manager-cainjector-559cdd8c87-v7znw\" (UID: \"0217870b-d6c5-4085-b929-1d81c3c860ce\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7znw" Jul 15 17:03:19.534187 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.534052 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0217870b-d6c5-4085-b929-1d81c3c860ce-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-v7znw\" (UID: \"0217870b-d6c5-4085-b929-1d81c3c860ce\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7znw" Jul 15 17:03:19.542409 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.542384 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0217870b-d6c5-4085-b929-1d81c3c860ce-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-v7znw\" (UID: \"0217870b-d6c5-4085-b929-1d81c3c860ce\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7znw" Jul 15 17:03:19.542605 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.542584 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdmnv\" (UniqueName: \"kubernetes.io/projected/0217870b-d6c5-4085-b929-1d81c3c860ce-kube-api-access-qdmnv\") pod \"cert-manager-cainjector-559cdd8c87-v7znw\" (UID: \"0217870b-d6c5-4085-b929-1d81c3c860ce\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7znw" Jul 15 17:03:19.627675 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.627656 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7znw" Jul 15 17:03:19.754527 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:19.754499 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-v7znw"] Jul 15 17:03:19.758002 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:03:19.757976 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0217870b_d6c5_4085_b929_1d81c3c860ce.slice/crio-ab3aeb57004ce369468037ad8455fa4a4e5a150bd922f3705cdf0e626a4ccc91 WatchSource:0}: Error finding container ab3aeb57004ce369468037ad8455fa4a4e5a150bd922f3705cdf0e626a4ccc91: Status 404 returned error can't find the container with id ab3aeb57004ce369468037ad8455fa4a4e5a150bd922f3705cdf0e626a4ccc91 Jul 15 17:03:20.220488 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:20.220453 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:20.325225 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:20.325174 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:20.352135 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:20.352102 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7znw" event={"ID":"0217870b-d6c5-4085-b929-1d81c3c860ce","Type":"ContainerStarted","Data":"c3773787a350746d3c62c84647a0ccdca9ee7822dbb511bfde74be9c98522b4f"} Jul 15 17:03:20.352135 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:20.352138 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7znw" event={"ID":"0217870b-d6c5-4085-b929-1d81c3c860ce","Type":"ContainerStarted","Data":"ab3aeb57004ce369468037ad8455fa4a4e5a150bd922f3705cdf0e626a4ccc91"} Jul 15 17:03:20.371578 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:20.371536 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7znw" podStartSLOduration=1.371522235 podStartE2EDuration="1.371522235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:03:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:03:20.370304115 +0000 UTC m=+250.612299340" watchObservedRunningTime="2026-07-15 17:03:20.371522235 +0000 UTC m=+250.613517461" Jul 15 17:03:20.537058 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:20.536973 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:20.635534 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:20.635502 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:21.336432 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.336404 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7q999" Jul 15 17:03:21.600638 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.600560 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns"] Jul 15 17:03:21.603804 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.603789 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns" Jul 15 17:03:21.606399 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.606378 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 17:03:21.606619 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.606605 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 17:03:21.606665 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.606605 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-9qnt9\"" Jul 15 17:03:21.614272 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.614253 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns"] Jul 15 17:03:21.651118 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.651090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8a8b2c7-60fb-4557-9814-fb2bb412b2e5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rbmns\" (UID: \"b8a8b2c7-60fb-4557-9814-fb2bb412b2e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns" Jul 15 17:03:21.651227 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.651150 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkxpr\" (UniqueName: \"kubernetes.io/projected/b8a8b2c7-60fb-4557-9814-fb2bb412b2e5-kube-api-access-wkxpr\") pod \"openshift-lws-operator-fd8ccff4c-rbmns\" (UID: \"b8a8b2c7-60fb-4557-9814-fb2bb412b2e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns" Jul 15 17:03:21.752411 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.752382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8a8b2c7-60fb-4557-9814-fb2bb412b2e5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rbmns\" (UID: \"b8a8b2c7-60fb-4557-9814-fb2bb412b2e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns" Jul 15 17:03:21.752527 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.752429 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkxpr\" (UniqueName: \"kubernetes.io/projected/b8a8b2c7-60fb-4557-9814-fb2bb412b2e5-kube-api-access-wkxpr\") pod \"openshift-lws-operator-fd8ccff4c-rbmns\" (UID: \"b8a8b2c7-60fb-4557-9814-fb2bb412b2e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns" Jul 15 17:03:21.752757 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.752735 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8a8b2c7-60fb-4557-9814-fb2bb412b2e5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rbmns\" (UID: \"b8a8b2c7-60fb-4557-9814-fb2bb412b2e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns" Jul 15 17:03:21.761108 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.761080 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkxpr\" (UniqueName: \"kubernetes.io/projected/b8a8b2c7-60fb-4557-9814-fb2bb412b2e5-kube-api-access-wkxpr\") pod \"openshift-lws-operator-fd8ccff4c-rbmns\" (UID: \"b8a8b2c7-60fb-4557-9814-fb2bb412b2e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns" Jul 15 17:03:21.913953 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:21.913885 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns" Jul 15 17:03:22.044166 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:22.044142 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns"] Jul 15 17:03:22.046423 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:03:22.046401 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8a8b2c7_60fb_4557_9814_fb2bb412b2e5.slice/crio-62cd22145d0f29ea6158f67f06c48b2b72a537ca91360f92294e29dfd4fe611b WatchSource:0}: Error finding container 62cd22145d0f29ea6158f67f06c48b2b72a537ca91360f92294e29dfd4fe611b: Status 404 returned error can't find the container with id 62cd22145d0f29ea6158f67f06c48b2b72a537ca91360f92294e29dfd4fe611b Jul 15 17:03:22.361907 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:22.361878 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns" event={"ID":"b8a8b2c7-60fb-4557-9814-fb2bb412b2e5","Type":"ContainerStarted","Data":"62cd22145d0f29ea6158f67f06c48b2b72a537ca91360f92294e29dfd4fe611b"} Jul 15 17:03:25.376057 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:25.376020 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns" event={"ID":"b8a8b2c7-60fb-4557-9814-fb2bb412b2e5","Type":"ContainerStarted","Data":"de0bb29656dc80ee4fcaf6d3c2ec560346ff8dd14f22720f1eabd9cc434e15e3"} Jul 15 17:03:25.394366 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:25.394312 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rbmns" podStartSLOduration=2.127987228 podStartE2EDuration="4.394292841s" totalImagesPullingTime="2.266305613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:03:25.392715079 +0000 UTC m=+255.634710305" watchObservedRunningTime="2026-07-15 17:03:25.394292841 +0000 UTC m=+255.636288067" Jul 15 17:03:26.048805 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:26.048771 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:26.152135 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:26.152104 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:27.909117 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:27.909081 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx"] Jul 15 17:03:27.931783 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:27.931755 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx"] Jul 15 17:03:27.931929 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:27.931870 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:27.934727 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:27.934706 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 17:03:27.935628 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:27.935604 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 17:03:27.935721 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:27.935606 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xjkzd\"" Jul 15 17:03:28.010095 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.010070 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:28.010242 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.010163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rhnt\" (UniqueName: \"kubernetes.io/projected/a30f1602-0807-40ba-be2a-304d680b885d-kube-api-access-4rhnt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:28.010242 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.010203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:28.068007 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.067975 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:28.111479 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.111453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rhnt\" (UniqueName: \"kubernetes.io/projected/a30f1602-0807-40ba-be2a-304d680b885d-kube-api-access-4rhnt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:28.111603 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.111490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:28.111603 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.111528 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:28.111885 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.111858 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:28.111927 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.111877 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:28.120136 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.120115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rhnt\" (UniqueName: \"kubernetes.io/projected/a30f1602-0807-40ba-be2a-304d680b885d-kube-api-access-4rhnt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:28.241166 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.241099 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:28.390297 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:28.390272 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx"] Jul 15 17:03:28.391553 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:03:28.391525 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda30f1602_0807_40ba_be2a_304d680b885d.slice/crio-64a0b3124da4b4aa9bc007fcdde6d0844ab4e7e011e4107c4b631191401b39ff WatchSource:0}: Error finding container 64a0b3124da4b4aa9bc007fcdde6d0844ab4e7e011e4107c4b631191401b39ff: Status 404 returned error can't find the container with id 64a0b3124da4b4aa9bc007fcdde6d0844ab4e7e011e4107c4b631191401b39ff Jul 15 17:03:29.395721 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:29.395635 2562 generic.go:356] "Generic (PLEG): container finished" podID="a30f1602-0807-40ba-be2a-304d680b885d" containerID="c58803211e4dfa48e73648579214de1bba1b58d31af55d79267cc05b3fba35a4" exitCode=0 Jul 15 17:03:29.395721 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:29.395680 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" event={"ID":"a30f1602-0807-40ba-be2a-304d680b885d","Type":"ContainerDied","Data":"c58803211e4dfa48e73648579214de1bba1b58d31af55d79267cc05b3fba35a4"} Jul 15 17:03:29.396092 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:29.395724 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" event={"ID":"a30f1602-0807-40ba-be2a-304d680b885d","Type":"ContainerStarted","Data":"64a0b3124da4b4aa9bc007fcdde6d0844ab4e7e011e4107c4b631191401b39ff"} Jul 15 17:03:30.402290 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:30.402261 2562 generic.go:356] "Generic (PLEG): container finished" podID="a30f1602-0807-40ba-be2a-304d680b885d" containerID="2eab5689bee11dc987fb68e505d751c8b0c392387fcd68224dc2b50fc487fe12" exitCode=0 Jul 15 17:03:30.402632 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:30.402337 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" event={"ID":"a30f1602-0807-40ba-be2a-304d680b885d","Type":"ContainerDied","Data":"2eab5689bee11dc987fb68e505d751c8b0c392387fcd68224dc2b50fc487fe12"} Jul 15 17:03:31.408851 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:31.408798 2562 generic.go:356] "Generic (PLEG): container finished" podID="a30f1602-0807-40ba-be2a-304d680b885d" containerID="8986640e30f00458add04badc90b984f6270f2f9cfbaaad94e37d43b9a7c1c30" exitCode=0 Jul 15 17:03:31.409203 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:31.408887 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" event={"ID":"a30f1602-0807-40ba-be2a-304d680b885d","Type":"ContainerDied","Data":"8986640e30f00458add04badc90b984f6270f2f9cfbaaad94e37d43b9a7c1c30"} Jul 15 17:03:32.544823 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:32.544800 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:32.650984 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:32.650954 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-bundle\") pod \"a30f1602-0807-40ba-be2a-304d680b885d\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " Jul 15 17:03:32.651157 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:32.651011 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-util\") pod \"a30f1602-0807-40ba-be2a-304d680b885d\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " Jul 15 17:03:32.651157 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:32.651039 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rhnt\" (UniqueName: \"kubernetes.io/projected/a30f1602-0807-40ba-be2a-304d680b885d-kube-api-access-4rhnt\") pod \"a30f1602-0807-40ba-be2a-304d680b885d\" (UID: \"a30f1602-0807-40ba-be2a-304d680b885d\") " Jul 15 17:03:32.651681 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:32.651640 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-bundle" (OuterVolumeSpecName: "bundle") pod "a30f1602-0807-40ba-be2a-304d680b885d" (UID: "a30f1602-0807-40ba-be2a-304d680b885d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:03:32.653261 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:32.653233 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a30f1602-0807-40ba-be2a-304d680b885d-kube-api-access-4rhnt" (OuterVolumeSpecName: "kube-api-access-4rhnt") pod "a30f1602-0807-40ba-be2a-304d680b885d" (UID: "a30f1602-0807-40ba-be2a-304d680b885d"). InnerVolumeSpecName "kube-api-access-4rhnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:03:32.656760 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:32.656734 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-util" (OuterVolumeSpecName: "util") pod "a30f1602-0807-40ba-be2a-304d680b885d" (UID: "a30f1602-0807-40ba-be2a-304d680b885d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:03:32.751804 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:32.751735 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-bundle\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:03:32.751804 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:32.751757 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a30f1602-0807-40ba-be2a-304d680b885d-util\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:03:32.751804 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:32.751766 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rhnt\" (UniqueName: \"kubernetes.io/projected/a30f1602-0807-40ba-be2a-304d680b885d-kube-api-access-4rhnt\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:03:33.420189 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:33.420166 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" Jul 15 17:03:33.420189 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:33.420175 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv79qx" event={"ID":"a30f1602-0807-40ba-be2a-304d680b885d","Type":"ContainerDied","Data":"64a0b3124da4b4aa9bc007fcdde6d0844ab4e7e011e4107c4b631191401b39ff"} Jul 15 17:03:33.420381 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:33.420214 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64a0b3124da4b4aa9bc007fcdde6d0844ab4e7e011e4107c4b631191401b39ff" Jul 15 17:03:36.280181 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.280150 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:36.382942 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.382915 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv"] Jul 15 17:03:36.383217 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.383206 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a30f1602-0807-40ba-be2a-304d680b885d" containerName="util" Jul 15 17:03:36.383258 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.383218 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a30f1602-0807-40ba-be2a-304d680b885d" containerName="util" Jul 15 17:03:36.383258 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.383233 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a30f1602-0807-40ba-be2a-304d680b885d" containerName="extract" Jul 15 17:03:36.383258 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.383239 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a30f1602-0807-40ba-be2a-304d680b885d" containerName="extract" Jul 15 17:03:36.383258 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.383247 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a30f1602-0807-40ba-be2a-304d680b885d" containerName="pull" Jul 15 17:03:36.383258 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.383252 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a30f1602-0807-40ba-be2a-304d680b885d" containerName="pull" Jul 15 17:03:36.383427 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.383303 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a30f1602-0807-40ba-be2a-304d680b885d" containerName="extract" Jul 15 17:03:36.386528 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.386512 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.390719 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.390698 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 17:03:36.390849 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.390744 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xzr2b\"" Jul 15 17:03:36.390849 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.390754 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 17:03:36.390849 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.390768 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 17:03:36.399714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.399694 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv"] Jul 15 17:03:36.418872 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.418827 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:36.482399 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.482365 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-metrics-cert\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.482544 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.482408 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vngx\" (UniqueName: \"kubernetes.io/projected/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-kube-api-access-4vngx\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.482544 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.482512 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-manager-config\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.482636 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.482575 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-cert\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.583526 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.583503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-metrics-cert\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.583658 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.583538 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vngx\" (UniqueName: \"kubernetes.io/projected/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-kube-api-access-4vngx\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.583724 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.583672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-manager-config\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.583777 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.583744 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-cert\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.586177 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.584791 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-manager-config\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.587092 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.587066 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-cert\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.591145 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.591124 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-metrics-cert\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.593024 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.593005 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vngx\" (UniqueName: \"kubernetes.io/projected/38bc0cec-2380-4b99-a0e1-71cb3e32ad37-kube-api-access-4vngx\") pod \"lws-controller-manager-6744c79bb-m9dfv\" (UID: \"38bc0cec-2380-4b99-a0e1-71cb3e32ad37\") " pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.695473 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.695443 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:36.832606 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:03:36.832569 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bc0cec_2380_4b99_a0e1_71cb3e32ad37.slice/crio-dc78dbad3f9fc32b532c16a507ac57f12fe6640771bada95c06ec3a66e29db52 WatchSource:0}: Error finding container dc78dbad3f9fc32b532c16a507ac57f12fe6640771bada95c06ec3a66e29db52: Status 404 returned error can't find the container with id dc78dbad3f9fc32b532c16a507ac57f12fe6640771bada95c06ec3a66e29db52 Jul 15 17:03:36.834004 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:36.833950 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv"] Jul 15 17:03:37.438282 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:37.438248 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" event={"ID":"38bc0cec-2380-4b99-a0e1-71cb3e32ad37","Type":"ContainerStarted","Data":"dc78dbad3f9fc32b532c16a507ac57f12fe6640771bada95c06ec3a66e29db52"} Jul 15 17:03:39.448307 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:39.448273 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" event={"ID":"38bc0cec-2380-4b99-a0e1-71cb3e32ad37","Type":"ContainerStarted","Data":"6f439eb695eb88ef7537ec4559aa7d2dbf87714a0f4a6dee02d236858cbf7067"} Jul 15 17:03:39.448642 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:39.448385 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:39.471400 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:39.471351 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" podStartSLOduration=1.8030043999999998 podStartE2EDuration="3.471337569s" totalImagesPullingTime="1.668333169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:03:39.469383235 +0000 UTC m=+269.711378471" watchObservedRunningTime="2026-07-15 17:03:39.471337569 +0000 UTC m=+269.713332795" Jul 15 17:03:44.013924 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:44.013861 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:44.109130 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:44.109087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:44.628581 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:44.628548 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:44.884317 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:44.884235 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:44.982501 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:44.982473 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:45.023639 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.023613 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj"] Jul 15 17:03:45.027049 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.027033 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.029683 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.029661 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 17:03:45.029808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.029663 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-z4dlf\"" Jul 15 17:03:45.029926 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.029909 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 17:03:45.029995 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.029966 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 17:03:45.030655 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.030638 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 17:03:45.043244 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.043223 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj"] Jul 15 17:03:45.052685 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.052647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53c2b68b-d7de-480a-b5df-38e853644702-tmp\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.052775 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.052699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/53c2b68b-d7de-480a-b5df-38e853644702-manifests\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.052824 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.052774 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53c2b68b-d7de-480a-b5df-38e853644702-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.052908 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.052890 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ttq2\" (UniqueName: \"kubernetes.io/projected/53c2b68b-d7de-480a-b5df-38e853644702-kube-api-access-7ttq2\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.052959 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.052929 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53c2b68b-d7de-480a-b5df-38e853644702-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.154200 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.154104 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53c2b68b-d7de-480a-b5df-38e853644702-tmp\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.154200 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.154153 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/53c2b68b-d7de-480a-b5df-38e853644702-manifests\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.154200 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.154194 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53c2b68b-d7de-480a-b5df-38e853644702-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.154469 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.154258 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ttq2\" (UniqueName: \"kubernetes.io/projected/53c2b68b-d7de-480a-b5df-38e853644702-kube-api-access-7ttq2\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.154469 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.154288 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53c2b68b-d7de-480a-b5df-38e853644702-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.154617 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.154592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/53c2b68b-d7de-480a-b5df-38e853644702-manifests\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.157130 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.157109 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53c2b68b-d7de-480a-b5df-38e853644702-tmp\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.157293 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.157269 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53c2b68b-d7de-480a-b5df-38e853644702-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.157362 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.157295 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53c2b68b-d7de-480a-b5df-38e853644702-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.167340 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.167317 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ttq2\" (UniqueName: \"kubernetes.io/projected/53c2b68b-d7de-480a-b5df-38e853644702-kube-api-access-7ttq2\") pod \"opendatahub-operator-controller-manager-7fdf764586-rsdsj\" (UID: \"53c2b68b-d7de-480a-b5df-38e853644702\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.223082 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.223040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:45.327244 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.327191 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:45.337157 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.337137 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:45.426348 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.426315 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:45.477136 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.477105 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj"] Jul 15 17:03:45.480289 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:03:45.480264 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c2b68b_d7de_480a_b5df_38e853644702.slice/crio-8c3cb6b40662835b6fc91f466aa288709e142a97249a65d82bd67873ad7caf95 WatchSource:0}: Error finding container 8c3cb6b40662835b6fc91f466aa288709e142a97249a65d82bd67873ad7caf95: Status 404 returned error can't find the container with id 8c3cb6b40662835b6fc91f466aa288709e142a97249a65d82bd67873ad7caf95 Jul 15 17:03:45.524140 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.524118 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:45.624266 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.624225 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:45.725648 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.725567 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:45.828354 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:45.828310 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:46.482909 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:46.482856 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" event={"ID":"53c2b68b-d7de-480a-b5df-38e853644702","Type":"ContainerStarted","Data":"8c3cb6b40662835b6fc91f466aa288709e142a97249a65d82bd67873ad7caf95"} Jul 15 17:03:48.493134 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:48.493093 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" event={"ID":"53c2b68b-d7de-480a-b5df-38e853644702","Type":"ContainerStarted","Data":"82d9a6fa4e7acf82ba7ddf8e9f1b10c2f5ebc3dde9eed997147725db93ea445f"} Jul 15 17:03:49.499278 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:49.499247 2562 generic.go:356] "Generic (PLEG): container finished" podID="53c2b68b-d7de-480a-b5df-38e853644702" containerID="82d9a6fa4e7acf82ba7ddf8e9f1b10c2f5ebc3dde9eed997147725db93ea445f" exitCode=0 Jul 15 17:03:49.499657 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:49.499285 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" event={"ID":"53c2b68b-d7de-480a-b5df-38e853644702","Type":"ContainerDied","Data":"82d9a6fa4e7acf82ba7ddf8e9f1b10c2f5ebc3dde9eed997147725db93ea445f"} Jul 15 17:03:50.455506 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:50.455471 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6744c79bb-m9dfv" Jul 15 17:03:50.507141 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:50.507108 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" event={"ID":"53c2b68b-d7de-480a-b5df-38e853644702","Type":"ContainerStarted","Data":"cefd81186ca3b9bc96f0380d0e78b31e357d277136d05a745dcfecf89e5ff483"} Jul 15 17:03:50.507537 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:50.507258 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:03:50.539335 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:50.539283 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" podStartSLOduration=2.29073767 podStartE2EDuration="6.539262986s" totalImagesPullingTime="3.253150735s" totalInitContainerRuntime="995.374581ms" isStatefulPod=false podCreationTimestamp="2026-07-15 17:03:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:03:50.535026681 +0000 UTC m=+280.777021923" watchObservedRunningTime="2026-07-15 17:03:50.539262986 +0000 UTC m=+280.781258212" Jul 15 17:03:56.621405 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:56.621370 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:56.713568 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:03:56.713526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:01.513695 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:01.513666 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-rsdsj" Jul 15 17:04:04.156823 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:04.156787 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:04.329517 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:04.329480 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:04.522660 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:04.522561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:04.814831 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:04.814745 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:05.099152 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:05.099118 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:05.195364 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:05.195330 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:05.272867 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:05.272810 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:05.371238 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:05.371133 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:06.011068 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.011027 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:06.142903 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.142863 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:06.159234 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.159203 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l"] Jul 15 17:04:06.163991 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.163972 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.166548 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.166527 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 17:04:06.166704 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.166682 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 17:04:06.166788 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.166735 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kp5jp\"" Jul 15 17:04:06.167071 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.167050 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 17:04:06.167206 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.167057 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 17:04:06.167280 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.167235 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 17:04:06.167280 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.167148 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 17:04:06.178356 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.178336 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l"] Jul 15 17:04:06.234928 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.234901 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-cacerts\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.235293 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.234937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-local-certs\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.235293 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.235072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.235293 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.235104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thcgd\" (UniqueName: \"kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-kube-api-access-thcgd\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.235293 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.235133 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-token\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.235293 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.235150 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.235293 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.235235 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.279701 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.279617 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:06.336591 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.336568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.336748 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.336597 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thcgd\" (UniqueName: \"kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-kube-api-access-thcgd\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.336748 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.336616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-token\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.336748 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.336632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.336748 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.336667 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.336748 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.336743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-cacerts\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.337020 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.336775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-local-certs\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.337368 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.337344 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.339437 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.339412 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-cacerts\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.339766 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.339742 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.339891 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.339791 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-local-certs\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.339891 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.339861 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.344811 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.344785 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thcgd\" (UniqueName: \"kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-kube-api-access-thcgd\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.345223 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.345201 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-token\") pod \"istiod-openshift-gateway-7574459656-m5k6l\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.475162 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.475126 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:06.618249 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:06.618212 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l"] Jul 15 17:04:06.620410 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:04:06.620382 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ef8c8f_f439_44ea_a2b7_3edfb7a7e33c.slice/crio-e91896cd9d02d2b460f4c8014be8ad29cc1b5e70d1bdba99c6e4b7ec8944ca20 WatchSource:0}: Error finding container e91896cd9d02d2b460f4c8014be8ad29cc1b5e70d1bdba99c6e4b7ec8944ca20: Status 404 returned error can't find the container with id e91896cd9d02d2b460f4c8014be8ad29cc1b5e70d1bdba99c6e4b7ec8944ca20 Jul 15 17:04:07.385441 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:07.385376 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:07.587573 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:07.587508 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" event={"ID":"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c","Type":"ContainerStarted","Data":"e91896cd9d02d2b460f4c8014be8ad29cc1b5e70d1bdba99c6e4b7ec8944ca20"} Jul 15 17:04:09.133409 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:09.133372 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:04:09.133684 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:09.133470 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:04:09.599484 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:09.599448 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" event={"ID":"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c","Type":"ContainerStarted","Data":"e0052a163c94a521ea572ac7aaf5ebafe54ebbf83ff319d4fbf602a977467390"} Jul 15 17:04:09.599642 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:09.599581 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:09.633013 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:09.632959 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" podStartSLOduration=1.12224337 podStartE2EDuration="3.632943501s" totalImagesPullingTime="2.510700131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:04:09.632004409 +0000 UTC m=+299.873999634" watchObservedRunningTime="2026-07-15 17:04:09.632943501 +0000 UTC m=+299.874938729" Jul 15 17:04:09.852402 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:09.852304 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:09.952574 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:09.952533 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:10.108745 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:10.108677 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 17:04:10.151376 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:10.151338 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:10.335915 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:10.335883 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:10.606079 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:10.606052 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:11.029169 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:11.029071 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:16.054119 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:16.054073 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:16.270326 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:16.270285 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:16.357418 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:16.357374 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:16.486561 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:16.486530 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:17.513644 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.513614 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l"] Jul 15 17:04:17.514086 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.513818 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" podUID="a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" containerName="discovery" containerID="cri-o://e0052a163c94a521ea572ac7aaf5ebafe54ebbf83ff319d4fbf602a977467390" gracePeriod=30 Jul 15 17:04:17.638657 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.638626 2562 generic.go:356] "Generic (PLEG): container finished" podID="a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" containerID="e0052a163c94a521ea572ac7aaf5ebafe54ebbf83ff319d4fbf602a977467390" exitCode=0 Jul 15 17:04:17.638796 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.638698 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" event={"ID":"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c","Type":"ContainerDied","Data":"e0052a163c94a521ea572ac7aaf5ebafe54ebbf83ff319d4fbf602a977467390"} Jul 15 17:04:17.771312 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.771246 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:17.839069 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.839020 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:17.938493 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.938465 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:17.943457 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.943440 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-cacerts\") pod \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " Jul 15 17:04:17.943531 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.943479 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-local-certs\") pod \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " Jul 15 17:04:17.943531 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.943507 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-token\") pod \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " Jul 15 17:04:17.943615 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.943556 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thcgd\" (UniqueName: \"kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-kube-api-access-thcgd\") pod \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " Jul 15 17:04:17.943615 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.943584 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-dns-cert\") pod \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " Jul 15 17:04:17.943706 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.943645 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-ca-configmap\") pod \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " Jul 15 17:04:17.943764 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.943711 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-kubeconfig\") pod \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\" (UID: \"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c\") " Jul 15 17:04:17.944142 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.944066 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" (UID: "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:04:17.946126 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.946073 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-local-certs" (OuterVolumeSpecName: "local-certs") pod "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" (UID: "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:04:17.946250 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.946153 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-kube-api-access-thcgd" (OuterVolumeSpecName: "kube-api-access-thcgd") pod "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" (UID: "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c"). InnerVolumeSpecName "kube-api-access-thcgd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:04:17.946315 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.946274 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" (UID: "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:04:17.946381 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.946361 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-cacerts" (OuterVolumeSpecName: "cacerts") pod "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" (UID: "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:04:17.946461 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.946443 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" (UID: "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:04:17.946573 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:17.946560 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" (UID: "a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:04:18.044371 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.044301 2562 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-cacerts\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:04:18.044371 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.044326 2562 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-local-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:04:18.044371 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.044335 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-token\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:04:18.044371 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.044343 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thcgd\" (UniqueName: \"kubernetes.io/projected/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-kube-api-access-thcgd\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:04:18.044371 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.044352 2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-dns-cert\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:04:18.044371 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.044360 2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-csr-ca-configmap\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:04:18.044371 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.044368 2562 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c-istio-kubeconfig\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:04:18.059579 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.059534 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:04:18.644848 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.644807 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" event={"ID":"a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c","Type":"ContainerDied","Data":"e91896cd9d02d2b460f4c8014be8ad29cc1b5e70d1bdba99c6e4b7ec8944ca20"} Jul 15 17:04:18.645239 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.644887 2562 scope.go:117] "RemoveContainer" containerID="e0052a163c94a521ea572ac7aaf5ebafe54ebbf83ff319d4fbf602a977467390" Jul 15 17:04:18.645239 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.644822 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l" Jul 15 17:04:18.665903 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.665874 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l"] Jul 15 17:04:18.671307 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:18.671283 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m5k6l"] Jul 15 17:04:20.251403 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:04:20.251369 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" path="/var/lib/kubelet/pods/a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c/volumes" Jul 15 17:05:05.721571 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.721448 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:05.818133 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.818086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:05.857525 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.857497 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frx2q"] Jul 15 17:05:05.857819 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.857807 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" containerName="discovery" Jul 15 17:05:05.857880 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.857823 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" containerName="discovery" Jul 15 17:05:05.857939 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.857928 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4ef8c8f-f439-44ea-a2b7-3edfb7a7e33c" containerName="discovery" Jul 15 17:05:05.860829 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.860813 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-frx2q" Jul 15 17:05:05.863472 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.863443 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 17:05:05.863580 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.863559 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 17:05:05.863624 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.863582 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2f77s\"" Jul 15 17:05:05.871614 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.871592 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frx2q"] Jul 15 17:05:05.918064 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.918028 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:05.947248 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:05.947223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p4jp\" (UniqueName: \"kubernetes.io/projected/1248809f-782e-4416-84c5-f0696f01af5e-kube-api-access-7p4jp\") pod \"kuadrant-operator-catalog-frx2q\" (UID: \"1248809f-782e-4416-84c5-f0696f01af5e\") " pod="kuadrant-system/kuadrant-operator-catalog-frx2q" Jul 15 17:05:06.048089 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.048018 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4jp\" (UniqueName: \"kubernetes.io/projected/1248809f-782e-4416-84c5-f0696f01af5e-kube-api-access-7p4jp\") pod \"kuadrant-operator-catalog-frx2q\" (UID: \"1248809f-782e-4416-84c5-f0696f01af5e\") " pod="kuadrant-system/kuadrant-operator-catalog-frx2q" Jul 15 17:05:06.056894 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.056869 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p4jp\" (UniqueName: \"kubernetes.io/projected/1248809f-782e-4416-84c5-f0696f01af5e-kube-api-access-7p4jp\") pod \"kuadrant-operator-catalog-frx2q\" (UID: \"1248809f-782e-4416-84c5-f0696f01af5e\") " pod="kuadrant-system/kuadrant-operator-catalog-frx2q" Jul 15 17:05:06.172073 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.172042 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-frx2q" Jul 15 17:05:06.217737 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.217702 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frx2q"] Jul 15 17:05:06.303799 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.303748 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frx2q"] Jul 15 17:05:06.305905 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:05:06.305873 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1248809f_782e_4416_84c5_f0696f01af5e.slice/crio-6530e3deeb206fb92ed10f6ecb290917abb7bc67ee6399350da084302ade8989 WatchSource:0}: Error finding container 6530e3deeb206fb92ed10f6ecb290917abb7bc67ee6399350da084302ade8989: Status 404 returned error can't find the container with id 6530e3deeb206fb92ed10f6ecb290917abb7bc67ee6399350da084302ade8989 Jul 15 17:05:06.307277 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.307258 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:05:06.428266 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.428230 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-84mk7"] Jul 15 17:05:06.432543 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.432527 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-84mk7" Jul 15 17:05:06.441260 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.441234 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-84mk7"] Jul 15 17:05:06.553880 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.553772 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zs77\" (UniqueName: \"kubernetes.io/projected/7b640a40-926d-4814-9982-23906b95e15c-kube-api-access-9zs77\") pod \"kuadrant-operator-catalog-84mk7\" (UID: \"7b640a40-926d-4814-9982-23906b95e15c\") " pod="kuadrant-system/kuadrant-operator-catalog-84mk7" Jul 15 17:05:06.653373 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.653341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:06.654329 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.654309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zs77\" (UniqueName: \"kubernetes.io/projected/7b640a40-926d-4814-9982-23906b95e15c-kube-api-access-9zs77\") pod \"kuadrant-operator-catalog-84mk7\" (UID: \"7b640a40-926d-4814-9982-23906b95e15c\") " pod="kuadrant-system/kuadrant-operator-catalog-84mk7" Jul 15 17:05:06.663437 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.663413 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zs77\" (UniqueName: \"kubernetes.io/projected/7b640a40-926d-4814-9982-23906b95e15c-kube-api-access-9zs77\") pod \"kuadrant-operator-catalog-84mk7\" (UID: \"7b640a40-926d-4814-9982-23906b95e15c\") " pod="kuadrant-system/kuadrant-operator-catalog-84mk7" Jul 15 17:05:06.742866 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.742821 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-84mk7" Jul 15 17:05:06.862320 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.862280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:06.865589 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.865563 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-frx2q" event={"ID":"1248809f-782e-4416-84c5-f0696f01af5e","Type":"ContainerStarted","Data":"6530e3deeb206fb92ed10f6ecb290917abb7bc67ee6399350da084302ade8989"} Jul 15 17:05:06.880498 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:06.880476 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-84mk7"] Jul 15 17:05:06.882499 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:05:06.882474 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b640a40_926d_4814_9982_23906b95e15c.slice/crio-6da6a56215185b00120f723d7aca72abdcfbf3f19faaf16fce88771128330016 WatchSource:0}: Error finding container 6da6a56215185b00120f723d7aca72abdcfbf3f19faaf16fce88771128330016: Status 404 returned error can't find the container with id 6da6a56215185b00120f723d7aca72abdcfbf3f19faaf16fce88771128330016 Jul 15 17:05:07.873347 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:07.873311 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-84mk7" event={"ID":"7b640a40-926d-4814-9982-23906b95e15c","Type":"ContainerStarted","Data":"6da6a56215185b00120f723d7aca72abdcfbf3f19faaf16fce88771128330016"} Jul 15 17:05:09.884948 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:09.884911 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-84mk7" event={"ID":"7b640a40-926d-4814-9982-23906b95e15c","Type":"ContainerStarted","Data":"afac95d82c854e95e7804a49671bbddb84373738f14536999e49e4bca07eb688"} Jul 15 17:05:09.906831 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:09.906783 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-84mk7" podStartSLOduration=1.65912038 podStartE2EDuration="3.906766917s" totalImagesPullingTime="2.247646537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:09.904361508 +0000 UTC m=+360.146356735" watchObservedRunningTime="2026-07-15 17:05:09.906766917 +0000 UTC m=+360.148762145" Jul 15 17:05:11.894853 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:11.894796 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-frx2q" event={"ID":"1248809f-782e-4416-84c5-f0696f01af5e","Type":"ContainerStarted","Data":"006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577"} Jul 15 17:05:11.895282 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:11.894863 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-frx2q" podUID="1248809f-782e-4416-84c5-f0696f01af5e" containerName="registry-server" containerID="cri-o://006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577" gracePeriod=2 Jul 15 17:05:11.918089 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:11.918050 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-frx2q" podStartSLOduration=2.367042293 podStartE2EDuration="6.918032555s" totalImagesPullingTime="4.550990262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:11.916184342 +0000 UTC m=+362.158179569" watchObservedRunningTime="2026-07-15 17:05:11.918032555 +0000 UTC m=+362.160027780" Jul 15 17:05:12.143051 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.143026 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-frx2q" Jul 15 17:05:12.306899 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.306780 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p4jp\" (UniqueName: \"kubernetes.io/projected/1248809f-782e-4416-84c5-f0696f01af5e-kube-api-access-7p4jp\") pod \"1248809f-782e-4416-84c5-f0696f01af5e\" (UID: \"1248809f-782e-4416-84c5-f0696f01af5e\") " Jul 15 17:05:12.309110 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.309072 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1248809f-782e-4416-84c5-f0696f01af5e-kube-api-access-7p4jp" (OuterVolumeSpecName: "kube-api-access-7p4jp") pod "1248809f-782e-4416-84c5-f0696f01af5e" (UID: "1248809f-782e-4416-84c5-f0696f01af5e"). InnerVolumeSpecName "kube-api-access-7p4jp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:05:12.407702 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.407669 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p4jp\" (UniqueName: \"kubernetes.io/projected/1248809f-782e-4416-84c5-f0696f01af5e-kube-api-access-7p4jp\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:12.900189 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.900153 2562 generic.go:356] "Generic (PLEG): container finished" podID="1248809f-782e-4416-84c5-f0696f01af5e" containerID="006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577" exitCode=0 Jul 15 17:05:12.900568 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.900200 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-frx2q" Jul 15 17:05:12.900568 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.900234 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-frx2q" event={"ID":"1248809f-782e-4416-84c5-f0696f01af5e","Type":"ContainerDied","Data":"006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577"} Jul 15 17:05:12.900568 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.900278 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-frx2q" event={"ID":"1248809f-782e-4416-84c5-f0696f01af5e","Type":"ContainerDied","Data":"6530e3deeb206fb92ed10f6ecb290917abb7bc67ee6399350da084302ade8989"} Jul 15 17:05:12.900568 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.900304 2562 scope.go:117] "RemoveContainer" containerID="006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577" Jul 15 17:05:12.908919 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.908900 2562 scope.go:117] "RemoveContainer" containerID="006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577" Jul 15 17:05:12.909156 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:05:12.909139 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577\": container with ID starting with 006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577 not found: ID does not exist" containerID="006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577" Jul 15 17:05:12.909221 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.909164 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577"} err="failed to get container status \"006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577\": rpc error: code = NotFound desc = could not find container \"006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577\": container with ID starting with 006163d9412046d527d990933e0ac3b37e0c81ff0631f915486e66685cbc5577 not found: ID does not exist" Jul 15 17:05:12.921438 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.921417 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frx2q"] Jul 15 17:05:12.925567 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:12.925545 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frx2q"] Jul 15 17:05:14.251107 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:14.251065 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1248809f-782e-4416-84c5-f0696f01af5e" path="/var/lib/kubelet/pods/1248809f-782e-4416-84c5-f0696f01af5e/volumes" Jul 15 17:05:15.670571 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:15.670536 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:15.784923 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:15.784888 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:15.853380 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:15.853354 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:16.430760 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:16.430700 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:16.515180 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:16.515138 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:16.743359 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:16.743270 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-84mk7" Jul 15 17:05:16.743359 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:16.743325 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-84mk7" Jul 15 17:05:16.766273 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:16.766249 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-84mk7" Jul 15 17:05:16.942069 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:16.942045 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-84mk7" Jul 15 17:05:21.870819 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.870771 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp"] Jul 15 17:05:21.871332 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.871115 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1248809f-782e-4416-84c5-f0696f01af5e" containerName="registry-server" Jul 15 17:05:21.871332 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.871126 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1248809f-782e-4416-84c5-f0696f01af5e" containerName="registry-server" Jul 15 17:05:21.871332 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.871188 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1248809f-782e-4416-84c5-f0696f01af5e" containerName="registry-server" Jul 15 17:05:21.875623 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.875596 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:21.876055 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.876034 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l9zz\" (UniqueName: \"kubernetes.io/projected/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-kube-api-access-6l9zz\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:21.876131 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.876096 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:21.876173 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.876138 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:21.878563 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.878547 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7clnc\"" Jul 15 17:05:21.885482 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.885464 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp"] Jul 15 17:05:21.976736 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.976710 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:21.976894 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.976743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:21.976894 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.976800 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l9zz\" (UniqueName: \"kubernetes.io/projected/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-kube-api-access-6l9zz\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:21.977111 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.977094 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:21.977148 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.977124 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:21.986195 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:21.986169 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l9zz\" (UniqueName: \"kubernetes.io/projected/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-kube-api-access-6l9zz\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:22.185236 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.185154 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:22.318859 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.318821 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp"] Jul 15 17:05:22.321209 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:05:22.321174 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb89bf294_eb84_46b3_8cbc_8c96cfae52c7.slice/crio-6b55b0464c46f99b7991af6b7d3c223bced24b9f173cd7b1a2b03e7d82e94a25 WatchSource:0}: Error finding container 6b55b0464c46f99b7991af6b7d3c223bced24b9f173cd7b1a2b03e7d82e94a25: Status 404 returned error can't find the container with id 6b55b0464c46f99b7991af6b7d3c223bced24b9f173cd7b1a2b03e7d82e94a25 Jul 15 17:05:22.665371 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.665340 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl"] Jul 15 17:05:22.668783 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.668767 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:22.678570 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.678549 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl"] Jul 15 17:05:22.681573 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.681553 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:22.681669 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.681589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hqbn\" (UniqueName: \"kubernetes.io/projected/31d68995-a464-4438-9bd6-a813f42c9156-kube-api-access-5hqbn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:22.681716 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.681673 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:22.782300 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.782276 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:22.782409 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.782322 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hqbn\" (UniqueName: \"kubernetes.io/projected/31d68995-a464-4438-9bd6-a813f42c9156-kube-api-access-5hqbn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:22.782409 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.782401 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:22.782629 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.782607 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:22.782714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.782694 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:22.791632 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.791600 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hqbn\" (UniqueName: \"kubernetes.io/projected/31d68995-a464-4438-9bd6-a813f42c9156-kube-api-access-5hqbn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:22.948466 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.948382 2562 generic.go:356] "Generic (PLEG): container finished" podID="b89bf294-eb84-46b3-8cbc-8c96cfae52c7" containerID="43caaf9d59f993770658a2f18d019e918e124395a57c454ee8b28665b6255c79" exitCode=0 Jul 15 17:05:22.948466 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.948450 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" event={"ID":"b89bf294-eb84-46b3-8cbc-8c96cfae52c7","Type":"ContainerDied","Data":"43caaf9d59f993770658a2f18d019e918e124395a57c454ee8b28665b6255c79"} Jul 15 17:05:22.948827 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.948499 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" event={"ID":"b89bf294-eb84-46b3-8cbc-8c96cfae52c7","Type":"ContainerStarted","Data":"6b55b0464c46f99b7991af6b7d3c223bced24b9f173cd7b1a2b03e7d82e94a25"} Jul 15 17:05:22.978346 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:22.978320 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:23.107147 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.107125 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl"] Jul 15 17:05:23.109136 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:05:23.109112 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d68995_a464_4438_9bd6_a813f42c9156.slice/crio-f51cddec316672662ee749ceb81108ebdaba31964e66b452a35bb119ef1697f4 WatchSource:0}: Error finding container f51cddec316672662ee749ceb81108ebdaba31964e66b452a35bb119ef1697f4: Status 404 returned error can't find the container with id f51cddec316672662ee749ceb81108ebdaba31964e66b452a35bb119ef1697f4 Jul 15 17:05:23.477931 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.477831 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4"] Jul 15 17:05:23.481333 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.481307 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:23.486872 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.486847 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n49jb\" (UniqueName: \"kubernetes.io/projected/b83c0568-810f-42bb-88c2-4fc3551f803b-kube-api-access-n49jb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:23.486995 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.486911 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:23.487063 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.487025 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:23.490927 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.490903 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4"] Jul 15 17:05:23.587567 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.587536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:23.587726 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.587577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n49jb\" (UniqueName: \"kubernetes.io/projected/b83c0568-810f-42bb-88c2-4fc3551f803b-kube-api-access-n49jb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:23.587726 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.587596 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:23.587973 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.587952 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:23.588045 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.587979 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:23.596704 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.596684 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n49jb\" (UniqueName: \"kubernetes.io/projected/b83c0568-810f-42bb-88c2-4fc3551f803b-kube-api-access-n49jb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:23.790645 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.790337 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:23.919258 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.919236 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4"] Jul 15 17:05:23.921056 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:05:23.921030 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb83c0568_810f_42bb_88c2_4fc3551f803b.slice/crio-345fe4658b15ad10eeec7c96cd015d3e98e3a7538ea8e50c1c0ee850f93d9ea4 WatchSource:0}: Error finding container 345fe4658b15ad10eeec7c96cd015d3e98e3a7538ea8e50c1c0ee850f93d9ea4: Status 404 returned error can't find the container with id 345fe4658b15ad10eeec7c96cd015d3e98e3a7538ea8e50c1c0ee850f93d9ea4 Jul 15 17:05:23.954793 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.954766 2562 generic.go:356] "Generic (PLEG): container finished" podID="31d68995-a464-4438-9bd6-a813f42c9156" containerID="519a67cf9922dac9bb043a2427aff6149484d6b51f1fd67214f13cde1b915cc8" exitCode=0 Jul 15 17:05:23.955164 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.954876 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" event={"ID":"31d68995-a464-4438-9bd6-a813f42c9156","Type":"ContainerDied","Data":"519a67cf9922dac9bb043a2427aff6149484d6b51f1fd67214f13cde1b915cc8"} Jul 15 17:05:23.955164 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.954930 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" event={"ID":"31d68995-a464-4438-9bd6-a813f42c9156","Type":"ContainerStarted","Data":"f51cddec316672662ee749ceb81108ebdaba31964e66b452a35bb119ef1697f4"} Jul 15 17:05:23.956062 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:23.956038 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" event={"ID":"b83c0568-810f-42bb-88c2-4fc3551f803b","Type":"ContainerStarted","Data":"345fe4658b15ad10eeec7c96cd015d3e98e3a7538ea8e50c1c0ee850f93d9ea4"} Jul 15 17:05:24.075205 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.075167 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4"] Jul 15 17:05:24.078957 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.078931 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:24.087734 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.087714 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4"] Jul 15 17:05:24.092093 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.092072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:24.092217 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.092100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f696n\" (UniqueName: \"kubernetes.io/projected/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-kube-api-access-f696n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:24.092217 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.092181 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:24.192602 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.192571 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:24.192745 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.192610 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:24.192745 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.192629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f696n\" (UniqueName: \"kubernetes.io/projected/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-kube-api-access-f696n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:24.192982 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.192964 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:24.193043 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.193026 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:24.201750 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.201722 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f696n\" (UniqueName: \"kubernetes.io/projected/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-kube-api-access-f696n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:24.388220 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.388141 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:24.517089 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.517064 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4"] Jul 15 17:05:24.518802 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:05:24.518778 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb41f6d0_3ec7_4d42_956e_c7a8819d3643.slice/crio-ab36ad54040fe068cc2cb7d2c5a8bf6fbbeb6503f36daadfdb2090a9b44e424e WatchSource:0}: Error finding container ab36ad54040fe068cc2cb7d2c5a8bf6fbbeb6503f36daadfdb2090a9b44e424e: Status 404 returned error can't find the container with id ab36ad54040fe068cc2cb7d2c5a8bf6fbbeb6503f36daadfdb2090a9b44e424e Jul 15 17:05:24.962465 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.962431 2562 generic.go:356] "Generic (PLEG): container finished" podID="cb41f6d0-3ec7-4d42-956e-c7a8819d3643" containerID="25409e794cef2d5ab577d3205bb26508e08e2b825c0b8041ce851ada96dd07ef" exitCode=0 Jul 15 17:05:24.962914 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.962536 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" event={"ID":"cb41f6d0-3ec7-4d42-956e-c7a8819d3643","Type":"ContainerDied","Data":"25409e794cef2d5ab577d3205bb26508e08e2b825c0b8041ce851ada96dd07ef"} Jul 15 17:05:24.962914 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.962582 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" event={"ID":"cb41f6d0-3ec7-4d42-956e-c7a8819d3643","Type":"ContainerStarted","Data":"ab36ad54040fe068cc2cb7d2c5a8bf6fbbeb6503f36daadfdb2090a9b44e424e"} Jul 15 17:05:24.964134 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.964112 2562 generic.go:356] "Generic (PLEG): container finished" podID="b83c0568-810f-42bb-88c2-4fc3551f803b" containerID="0be971ae632aaa58f72d9362f07529b0b6b3c77075c65a7e99f7aa2871c54fff" exitCode=0 Jul 15 17:05:24.964273 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:24.964184 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" event={"ID":"b83c0568-810f-42bb-88c2-4fc3551f803b","Type":"ContainerDied","Data":"0be971ae632aaa58f72d9362f07529b0b6b3c77075c65a7e99f7aa2871c54fff"} Jul 15 17:05:25.970683 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:25.970657 2562 generic.go:356] "Generic (PLEG): container finished" podID="b83c0568-810f-42bb-88c2-4fc3551f803b" containerID="f420578f44737454e61425d25a448ddd5bae3c89a81a7e18297283063adf9fea" exitCode=0 Jul 15 17:05:25.970980 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:25.970743 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" event={"ID":"b83c0568-810f-42bb-88c2-4fc3551f803b","Type":"ContainerDied","Data":"f420578f44737454e61425d25a448ddd5bae3c89a81a7e18297283063adf9fea"} Jul 15 17:05:26.977457 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:26.977430 2562 generic.go:356] "Generic (PLEG): container finished" podID="cb41f6d0-3ec7-4d42-956e-c7a8819d3643" containerID="e814b78b3c9bdfad452fdf865af559de459f9327f2e72e69df2f6e9e05614142" exitCode=0 Jul 15 17:05:26.977828 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:26.977518 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" event={"ID":"cb41f6d0-3ec7-4d42-956e-c7a8819d3643","Type":"ContainerDied","Data":"e814b78b3c9bdfad452fdf865af559de459f9327f2e72e69df2f6e9e05614142"} Jul 15 17:05:26.985299 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:26.985274 2562 generic.go:356] "Generic (PLEG): container finished" podID="b89bf294-eb84-46b3-8cbc-8c96cfae52c7" containerID="14a855b816f6d80061d58aa03e21d8e735c5909c6cca71a8dc1b0fb1c29a306d" exitCode=0 Jul 15 17:05:26.985411 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:26.985359 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" event={"ID":"b89bf294-eb84-46b3-8cbc-8c96cfae52c7","Type":"ContainerDied","Data":"14a855b816f6d80061d58aa03e21d8e735c5909c6cca71a8dc1b0fb1c29a306d"} Jul 15 17:05:26.988007 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:26.987932 2562 generic.go:356] "Generic (PLEG): container finished" podID="b83c0568-810f-42bb-88c2-4fc3551f803b" containerID="e8358fa12b9506114daec280baf1e71646cd8744bfa848021186004ef2a03b6a" exitCode=0 Jul 15 17:05:26.988007 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:26.987987 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" event={"ID":"b83c0568-810f-42bb-88c2-4fc3551f803b","Type":"ContainerDied","Data":"e8358fa12b9506114daec280baf1e71646cd8744bfa848021186004ef2a03b6a"} Jul 15 17:05:27.067057 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:27.067025 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:27.169068 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:27.169022 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:27.263400 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:27.263371 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:27.994705 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:27.994677 2562 generic.go:356] "Generic (PLEG): container finished" podID="b89bf294-eb84-46b3-8cbc-8c96cfae52c7" containerID="0831ecaa25bb73721cc147a81f015f4b33ce3e00772d6ad63c514b243a87e302" exitCode=0 Jul 15 17:05:27.995103 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:27.994747 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" event={"ID":"b89bf294-eb84-46b3-8cbc-8c96cfae52c7","Type":"ContainerDied","Data":"0831ecaa25bb73721cc147a81f015f4b33ce3e00772d6ad63c514b243a87e302"} Jul 15 17:05:27.996419 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:27.996399 2562 generic.go:356] "Generic (PLEG): container finished" podID="31d68995-a464-4438-9bd6-a813f42c9156" containerID="c3711fab6600534fbaf8decdb57acd34a321abb15f76b45bd66ed7dba4233484" exitCode=0 Jul 15 17:05:27.996511 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:27.996474 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" event={"ID":"31d68995-a464-4438-9bd6-a813f42c9156","Type":"ContainerDied","Data":"c3711fab6600534fbaf8decdb57acd34a321abb15f76b45bd66ed7dba4233484"} Jul 15 17:05:27.998541 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:27.998522 2562 generic.go:356] "Generic (PLEG): container finished" podID="cb41f6d0-3ec7-4d42-956e-c7a8819d3643" containerID="50d1aaf7376f436915f97e2b26833dcf8a429299c0b1aa3fd1e45b89c1b694b8" exitCode=0 Jul 15 17:05:27.998633 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:27.998610 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" event={"ID":"cb41f6d0-3ec7-4d42-956e-c7a8819d3643","Type":"ContainerDied","Data":"50d1aaf7376f436915f97e2b26833dcf8a429299c0b1aa3fd1e45b89c1b694b8"} Jul 15 17:05:28.133952 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:28.133930 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:28.221691 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:28.221666 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n49jb\" (UniqueName: \"kubernetes.io/projected/b83c0568-810f-42bb-88c2-4fc3551f803b-kube-api-access-n49jb\") pod \"b83c0568-810f-42bb-88c2-4fc3551f803b\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " Jul 15 17:05:28.221804 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:28.221709 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-util\") pod \"b83c0568-810f-42bb-88c2-4fc3551f803b\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " Jul 15 17:05:28.221804 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:28.221727 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-bundle\") pod \"b83c0568-810f-42bb-88c2-4fc3551f803b\" (UID: \"b83c0568-810f-42bb-88c2-4fc3551f803b\") " Jul 15 17:05:28.222190 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:28.222163 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-bundle" (OuterVolumeSpecName: "bundle") pod "b83c0568-810f-42bb-88c2-4fc3551f803b" (UID: "b83c0568-810f-42bb-88c2-4fc3551f803b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:28.223811 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:28.223785 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b83c0568-810f-42bb-88c2-4fc3551f803b-kube-api-access-n49jb" (OuterVolumeSpecName: "kube-api-access-n49jb") pod "b83c0568-810f-42bb-88c2-4fc3551f803b" (UID: "b83c0568-810f-42bb-88c2-4fc3551f803b"). InnerVolumeSpecName "kube-api-access-n49jb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:05:28.227238 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:28.227215 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-util" (OuterVolumeSpecName: "util") pod "b83c0568-810f-42bb-88c2-4fc3551f803b" (UID: "b83c0568-810f-42bb-88c2-4fc3551f803b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:28.322868 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:28.322820 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n49jb\" (UniqueName: \"kubernetes.io/projected/b83c0568-810f-42bb-88c2-4fc3551f803b-kube-api-access-n49jb\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:28.322868 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:28.322866 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-util\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:28.323013 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:28.322875 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b83c0568-810f-42bb-88c2-4fc3551f803b-bundle\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:29.005416 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.005389 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" Jul 15 17:05:29.005865 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.005422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4" event={"ID":"b83c0568-810f-42bb-88c2-4fc3551f803b","Type":"ContainerDied","Data":"345fe4658b15ad10eeec7c96cd015d3e98e3a7538ea8e50c1c0ee850f93d9ea4"} Jul 15 17:05:29.005865 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.005464 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="345fe4658b15ad10eeec7c96cd015d3e98e3a7538ea8e50c1c0ee850f93d9ea4" Jul 15 17:05:29.007521 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.007430 2562 generic.go:356] "Generic (PLEG): container finished" podID="31d68995-a464-4438-9bd6-a813f42c9156" containerID="89e258f3e12e4c30880404b77a2ad6a29b8e9c233b7fda22846bec61438ec90f" exitCode=0 Jul 15 17:05:29.007521 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.007459 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" event={"ID":"31d68995-a464-4438-9bd6-a813f42c9156","Type":"ContainerDied","Data":"89e258f3e12e4c30880404b77a2ad6a29b8e9c233b7fda22846bec61438ec90f"} Jul 15 17:05:29.166988 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.166967 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:29.170265 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.170247 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:29.228692 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.228670 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-bundle\") pod \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " Jul 15 17:05:29.228870 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.228709 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-bundle\") pod \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " Jul 15 17:05:29.228870 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.228728 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-util\") pod \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " Jul 15 17:05:29.228870 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.228748 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-util\") pod \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " Jul 15 17:05:29.228870 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.228784 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f696n\" (UniqueName: \"kubernetes.io/projected/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-kube-api-access-f696n\") pod \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\" (UID: \"cb41f6d0-3ec7-4d42-956e-c7a8819d3643\") " Jul 15 17:05:29.228870 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.228826 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6l9zz\" (UniqueName: \"kubernetes.io/projected/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-kube-api-access-6l9zz\") pod \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\" (UID: \"b89bf294-eb84-46b3-8cbc-8c96cfae52c7\") " Jul 15 17:05:29.229212 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.229184 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-bundle" (OuterVolumeSpecName: "bundle") pod "cb41f6d0-3ec7-4d42-956e-c7a8819d3643" (UID: "cb41f6d0-3ec7-4d42-956e-c7a8819d3643"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:29.229345 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.229317 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-bundle" (OuterVolumeSpecName: "bundle") pod "b89bf294-eb84-46b3-8cbc-8c96cfae52c7" (UID: "b89bf294-eb84-46b3-8cbc-8c96cfae52c7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:29.231539 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.231503 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-kube-api-access-f696n" (OuterVolumeSpecName: "kube-api-access-f696n") pod "cb41f6d0-3ec7-4d42-956e-c7a8819d3643" (UID: "cb41f6d0-3ec7-4d42-956e-c7a8819d3643"). InnerVolumeSpecName "kube-api-access-f696n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:05:29.231667 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.231647 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-kube-api-access-6l9zz" (OuterVolumeSpecName: "kube-api-access-6l9zz") pod "b89bf294-eb84-46b3-8cbc-8c96cfae52c7" (UID: "b89bf294-eb84-46b3-8cbc-8c96cfae52c7"). InnerVolumeSpecName "kube-api-access-6l9zz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:05:29.234724 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.234700 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-util" (OuterVolumeSpecName: "util") pod "b89bf294-eb84-46b3-8cbc-8c96cfae52c7" (UID: "b89bf294-eb84-46b3-8cbc-8c96cfae52c7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:29.235739 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.235718 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-util" (OuterVolumeSpecName: "util") pod "cb41f6d0-3ec7-4d42-956e-c7a8819d3643" (UID: "cb41f6d0-3ec7-4d42-956e-c7a8819d3643"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:29.330313 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.330289 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f696n\" (UniqueName: \"kubernetes.io/projected/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-kube-api-access-f696n\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:29.330313 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.330314 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6l9zz\" (UniqueName: \"kubernetes.io/projected/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-kube-api-access-6l9zz\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:29.330489 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.330327 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-bundle\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:29.330489 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.330339 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-bundle\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:29.330489 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.330349 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b89bf294-eb84-46b3-8cbc-8c96cfae52c7-util\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:29.330489 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:29.330358 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb41f6d0-3ec7-4d42-956e-c7a8819d3643-util\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:30.013791 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.013764 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" Jul 15 17:05:30.014212 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.013760 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4" event={"ID":"cb41f6d0-3ec7-4d42-956e-c7a8819d3643","Type":"ContainerDied","Data":"ab36ad54040fe068cc2cb7d2c5a8bf6fbbeb6503f36daadfdb2090a9b44e424e"} Jul 15 17:05:30.014212 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.013893 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab36ad54040fe068cc2cb7d2c5a8bf6fbbeb6503f36daadfdb2090a9b44e424e" Jul 15 17:05:30.015677 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.015659 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" Jul 15 17:05:30.015810 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.015694 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp" event={"ID":"b89bf294-eb84-46b3-8cbc-8c96cfae52c7","Type":"ContainerDied","Data":"6b55b0464c46f99b7991af6b7d3c223bced24b9f173cd7b1a2b03e7d82e94a25"} Jul 15 17:05:30.015810 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.015724 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b55b0464c46f99b7991af6b7d3c223bced24b9f173cd7b1a2b03e7d82e94a25" Jul 15 17:05:30.143302 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.143276 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:30.237913 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.237888 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hqbn\" (UniqueName: \"kubernetes.io/projected/31d68995-a464-4438-9bd6-a813f42c9156-kube-api-access-5hqbn\") pod \"31d68995-a464-4438-9bd6-a813f42c9156\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " Jul 15 17:05:30.238056 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.237934 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-bundle\") pod \"31d68995-a464-4438-9bd6-a813f42c9156\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " Jul 15 17:05:30.238056 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.237961 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-util\") pod \"31d68995-a464-4438-9bd6-a813f42c9156\" (UID: \"31d68995-a464-4438-9bd6-a813f42c9156\") " Jul 15 17:05:30.238564 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.238540 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-bundle" (OuterVolumeSpecName: "bundle") pod "31d68995-a464-4438-9bd6-a813f42c9156" (UID: "31d68995-a464-4438-9bd6-a813f42c9156"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:30.240173 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.240150 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31d68995-a464-4438-9bd6-a813f42c9156-kube-api-access-5hqbn" (OuterVolumeSpecName: "kube-api-access-5hqbn") pod "31d68995-a464-4438-9bd6-a813f42c9156" (UID: "31d68995-a464-4438-9bd6-a813f42c9156"). InnerVolumeSpecName "kube-api-access-5hqbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:05:30.242807 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.242784 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-util" (OuterVolumeSpecName: "util") pod "31d68995-a464-4438-9bd6-a813f42c9156" (UID: "31d68995-a464-4438-9bd6-a813f42c9156"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:05:30.339078 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.339045 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-util\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:30.339078 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.339074 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hqbn\" (UniqueName: \"kubernetes.io/projected/31d68995-a464-4438-9bd6-a813f42c9156-kube-api-access-5hqbn\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:30.339300 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:30.339087 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31d68995-a464-4438-9bd6-a813f42c9156-bundle\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:05:31.022252 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:31.022220 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" Jul 15 17:05:31.022651 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:31.022225 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl" event={"ID":"31d68995-a464-4438-9bd6-a813f42c9156","Type":"ContainerDied","Data":"f51cddec316672662ee749ceb81108ebdaba31964e66b452a35bb119ef1697f4"} Jul 15 17:05:31.022651 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:31.022349 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f51cddec316672662ee749ceb81108ebdaba31964e66b452a35bb119ef1697f4" Jul 15 17:05:36.269101 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:36.269065 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:38.926444 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926416 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2"] Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926729 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b83c0568-810f-42bb-88c2-4fc3551f803b" containerName="util" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926740 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b83c0568-810f-42bb-88c2-4fc3551f803b" containerName="util" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926747 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b89bf294-eb84-46b3-8cbc-8c96cfae52c7" containerName="pull" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926752 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b89bf294-eb84-46b3-8cbc-8c96cfae52c7" containerName="pull" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926762 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb41f6d0-3ec7-4d42-956e-c7a8819d3643" containerName="pull" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926767 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb41f6d0-3ec7-4d42-956e-c7a8819d3643" containerName="pull" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926774 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b89bf294-eb84-46b3-8cbc-8c96cfae52c7" containerName="extract" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926779 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b89bf294-eb84-46b3-8cbc-8c96cfae52c7" containerName="extract" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926788 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31d68995-a464-4438-9bd6-a813f42c9156" containerName="util" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926793 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="31d68995-a464-4438-9bd6-a813f42c9156" containerName="util" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926800 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31d68995-a464-4438-9bd6-a813f42c9156" containerName="extract" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926805 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="31d68995-a464-4438-9bd6-a813f42c9156" containerName="extract" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926814 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b89bf294-eb84-46b3-8cbc-8c96cfae52c7" containerName="util" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926819 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b89bf294-eb84-46b3-8cbc-8c96cfae52c7" containerName="util" Jul 15 17:05:38.926817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926824 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb41f6d0-3ec7-4d42-956e-c7a8819d3643" containerName="util" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926829 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb41f6d0-3ec7-4d42-956e-c7a8819d3643" containerName="util" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926848 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31d68995-a464-4438-9bd6-a813f42c9156" containerName="pull" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926855 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="31d68995-a464-4438-9bd6-a813f42c9156" containerName="pull" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926865 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb41f6d0-3ec7-4d42-956e-c7a8819d3643" containerName="extract" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926873 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb41f6d0-3ec7-4d42-956e-c7a8819d3643" containerName="extract" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926885 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b83c0568-810f-42bb-88c2-4fc3551f803b" containerName="extract" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926889 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b83c0568-810f-42bb-88c2-4fc3551f803b" containerName="extract" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926896 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b83c0568-810f-42bb-88c2-4fc3551f803b" containerName="pull" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926900 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b83c0568-810f-42bb-88c2-4fc3551f803b" containerName="pull" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926951 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b89bf294-eb84-46b3-8cbc-8c96cfae52c7" containerName="extract" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926959 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb41f6d0-3ec7-4d42-956e-c7a8819d3643" containerName="extract" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926965 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b83c0568-810f-42bb-88c2-4fc3551f803b" containerName="extract" Jul 15 17:05:38.927296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.926972 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="31d68995-a464-4438-9bd6-a813f42c9156" containerName="extract" Jul 15 17:05:38.934655 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.934640 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:05:38.937524 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.937505 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-tf8c4\"" Jul 15 17:05:38.949971 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:38.949950 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2"] Jul 15 17:05:39.011724 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:39.011697 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwwvn\" (UniqueName: \"kubernetes.io/projected/5e8d4224-9795-46ed-a414-2a84f99dc90a-kube-api-access-dwwvn\") pod \"limitador-operator-controller-manager-859d7b78bd-76dt2\" (UID: \"5e8d4224-9795-46ed-a414-2a84f99dc90a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:05:39.113064 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:39.113036 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwwvn\" (UniqueName: \"kubernetes.io/projected/5e8d4224-9795-46ed-a414-2a84f99dc90a-kube-api-access-dwwvn\") pod \"limitador-operator-controller-manager-859d7b78bd-76dt2\" (UID: \"5e8d4224-9795-46ed-a414-2a84f99dc90a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:05:39.145824 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:39.145799 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwwvn\" (UniqueName: \"kubernetes.io/projected/5e8d4224-9795-46ed-a414-2a84f99dc90a-kube-api-access-dwwvn\") pod \"limitador-operator-controller-manager-859d7b78bd-76dt2\" (UID: \"5e8d4224-9795-46ed-a414-2a84f99dc90a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:05:39.244576 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:39.244490 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:05:39.391363 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:39.391335 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2"] Jul 15 17:05:39.393785 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:05:39.393758 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e8d4224_9795_46ed_a414_2a84f99dc90a.slice/crio-c49cc61ecadc4ea0cf08f2bc147ba60bd0070e4ef4ae86476cd6f25c02e05bb0 WatchSource:0}: Error finding container c49cc61ecadc4ea0cf08f2bc147ba60bd0070e4ef4ae86476cd6f25c02e05bb0: Status 404 returned error can't find the container with id c49cc61ecadc4ea0cf08f2bc147ba60bd0070e4ef4ae86476cd6f25c02e05bb0 Jul 15 17:05:39.936002 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:39.935963 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:40.071101 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:40.071067 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" event={"ID":"5e8d4224-9795-46ed-a414-2a84f99dc90a","Type":"ContainerStarted","Data":"c49cc61ecadc4ea0cf08f2bc147ba60bd0070e4ef4ae86476cd6f25c02e05bb0"} Jul 15 17:05:42.082936 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:42.082900 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" event={"ID":"5e8d4224-9795-46ed-a414-2a84f99dc90a","Type":"ContainerStarted","Data":"4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818"} Jul 15 17:05:42.083282 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:42.083123 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:05:42.108433 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:42.108388 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" podStartSLOduration=1.924160327 podStartE2EDuration="4.108372682s" totalImagesPullingTime="2.184212355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:42.105316453 +0000 UTC m=+392.347311679" watchObservedRunningTime="2026-07-15 17:05:42.108372682 +0000 UTC m=+392.350367907" Jul 15 17:05:44.063398 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.063364 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww"] Jul 15 17:05:44.066823 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.066803 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:05:44.069421 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.069402 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9jw68\"" Jul 15 17:05:44.081323 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.081291 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww"] Jul 15 17:05:44.158900 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.158873 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcf8x\" (UniqueName: \"kubernetes.io/projected/2639457a-bba8-4fdd-a7d6-2759454c5235-kube-api-access-vcf8x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" (UID: \"2639457a-bba8-4fdd-a7d6-2759454c5235\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:05:44.159046 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.158916 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2639457a-bba8-4fdd-a7d6-2759454c5235-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" (UID: \"2639457a-bba8-4fdd-a7d6-2759454c5235\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:05:44.259960 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.259931 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcf8x\" (UniqueName: \"kubernetes.io/projected/2639457a-bba8-4fdd-a7d6-2759454c5235-kube-api-access-vcf8x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" (UID: \"2639457a-bba8-4fdd-a7d6-2759454c5235\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:05:44.260128 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.259975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2639457a-bba8-4fdd-a7d6-2759454c5235-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" (UID: \"2639457a-bba8-4fdd-a7d6-2759454c5235\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:05:44.260319 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.260304 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2639457a-bba8-4fdd-a7d6-2759454c5235-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" (UID: \"2639457a-bba8-4fdd-a7d6-2759454c5235\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:05:44.271943 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.271909 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcf8x\" (UniqueName: \"kubernetes.io/projected/2639457a-bba8-4fdd-a7d6-2759454c5235-kube-api-access-vcf8x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" (UID: \"2639457a-bba8-4fdd-a7d6-2759454c5235\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:05:44.377316 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.377286 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:05:44.459643 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.459611 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:44.515335 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:44.515310 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww"] Jul 15 17:05:44.517172 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:05:44.517146 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2639457a_bba8_4fdd_a7d6_2759454c5235.slice/crio-f934aeaa0ecaf0f6c52095a06acf62fcb6ee0f4f88eff16876745207366a5979 WatchSource:0}: Error finding container f934aeaa0ecaf0f6c52095a06acf62fcb6ee0f4f88eff16876745207366a5979: Status 404 returned error can't find the container with id f934aeaa0ecaf0f6c52095a06acf62fcb6ee0f4f88eff16876745207366a5979 Jul 15 17:05:45.058587 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:45.058553 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:45.103102 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:45.103066 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" event={"ID":"2639457a-bba8-4fdd-a7d6-2759454c5235","Type":"ContainerStarted","Data":"f934aeaa0ecaf0f6c52095a06acf62fcb6ee0f4f88eff16876745207366a5979"} Jul 15 17:05:45.515411 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:45.515370 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:47.257403 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:47.257371 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:47.654370 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:47.654339 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k"] Jul 15 17:05:47.657882 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:47.657863 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k" Jul 15 17:05:47.660922 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:47.660903 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-flgfz\"" Jul 15 17:05:47.667660 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:47.667638 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k"] Jul 15 17:05:47.798140 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:47.798111 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztp4h\" (UniqueName: \"kubernetes.io/projected/c06f6746-1f15-43bb-811f-84bcb0e5afdf-kube-api-access-ztp4h\") pod \"authorino-operator-75cc8d5cb7-vjz6k\" (UID: \"c06f6746-1f15-43bb-811f-84bcb0e5afdf\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k" Jul 15 17:05:47.905305 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:47.905211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztp4h\" (UniqueName: \"kubernetes.io/projected/c06f6746-1f15-43bb-811f-84bcb0e5afdf-kube-api-access-ztp4h\") pod \"authorino-operator-75cc8d5cb7-vjz6k\" (UID: \"c06f6746-1f15-43bb-811f-84bcb0e5afdf\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k" Jul 15 17:05:47.915576 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:47.915522 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztp4h\" (UniqueName: \"kubernetes.io/projected/c06f6746-1f15-43bb-811f-84bcb0e5afdf-kube-api-access-ztp4h\") pod \"authorino-operator-75cc8d5cb7-vjz6k\" (UID: \"c06f6746-1f15-43bb-811f-84bcb0e5afdf\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k" Jul 15 17:05:47.969070 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:47.969040 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k" Jul 15 17:05:48.118765 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:48.118633 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k"] Jul 15 17:05:48.121135 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:05:48.121103 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc06f6746_1f15_43bb_811f_84bcb0e5afdf.slice/crio-46292f5cd94a570f2ddd3fca5bf15196ee69937e29b4b9709cce094171e274ae WatchSource:0}: Error finding container 46292f5cd94a570f2ddd3fca5bf15196ee69937e29b4b9709cce094171e274ae: Status 404 returned error can't find the container with id 46292f5cd94a570f2ddd3fca5bf15196ee69937e29b4b9709cce094171e274ae Jul 15 17:05:48.922658 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:48.922628 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:49.129343 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:49.129309 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k" event={"ID":"c06f6746-1f15-43bb-811f-84bcb0e5afdf","Type":"ContainerStarted","Data":"46292f5cd94a570f2ddd3fca5bf15196ee69937e29b4b9709cce094171e274ae"} Jul 15 17:05:49.740033 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:49.739999 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:49.907209 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:49.907154 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:51.751175 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:51.751127 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:52.148301 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:52.148256 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k" event={"ID":"c06f6746-1f15-43bb-811f-84bcb0e5afdf","Type":"ContainerStarted","Data":"41edf7e1cb75afd2b1f1c9c478774a2b1aa750b7a1288d92b029c8b293d3b99a"} Jul 15 17:05:52.148481 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:52.148359 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k" Jul 15 17:05:52.149962 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:52.149936 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" event={"ID":"2639457a-bba8-4fdd-a7d6-2759454c5235","Type":"ContainerStarted","Data":"442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704"} Jul 15 17:05:52.150066 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:52.149985 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:05:52.180069 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:52.180012 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k" podStartSLOduration=1.399393701 podStartE2EDuration="5.179994396s" totalImagesPullingTime="3.780600695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:52.179522497 +0000 UTC m=+402.421517726" watchObservedRunningTime="2026-07-15 17:05:52.179994396 +0000 UTC m=+402.421989623" Jul 15 17:05:52.219658 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:52.219620 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" podStartSLOduration=0.823537516 podStartE2EDuration="8.219606061s" totalImagesPullingTime="7.396068545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:52.218446936 +0000 UTC m=+402.460442173" watchObservedRunningTime="2026-07-15 17:05:52.219606061 +0000 UTC m=+402.461601287" Jul 15 17:05:53.089743 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:53.089711 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:05:53.105670 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:53.105638 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:53.207611 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:53.207455 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:54.914791 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:54.914758 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs"] Jul 15 17:05:54.918667 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:54.918648 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" Jul 15 17:05:54.929399 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:54.929376 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7clnc\"" Jul 15 17:05:54.929532 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:54.929408 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 15 17:05:54.929532 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:54.929443 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 15 17:05:54.936519 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:54.936490 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs"] Jul 15 17:05:54.952278 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:54.952237 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:55.066058 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.066017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqllj\" (UniqueName: \"kubernetes.io/projected/f72d49b6-73af-48a0-8a20-c84388cfd160-kube-api-access-sqllj\") pod \"kuadrant-console-plugin-66b5774985-q9ncs\" (UID: \"f72d49b6-73af-48a0-8a20-c84388cfd160\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" Jul 15 17:05:55.066058 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.066058 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f72d49b6-73af-48a0-8a20-c84388cfd160-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q9ncs\" (UID: \"f72d49b6-73af-48a0-8a20-c84388cfd160\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" Jul 15 17:05:55.066262 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.066106 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f72d49b6-73af-48a0-8a20-c84388cfd160-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q9ncs\" (UID: \"f72d49b6-73af-48a0-8a20-c84388cfd160\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" Jul 15 17:05:55.100705 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.100669 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:55.167124 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.167047 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f72d49b6-73af-48a0-8a20-c84388cfd160-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q9ncs\" (UID: \"f72d49b6-73af-48a0-8a20-c84388cfd160\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" Jul 15 17:05:55.167124 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.167117 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqllj\" (UniqueName: \"kubernetes.io/projected/f72d49b6-73af-48a0-8a20-c84388cfd160-kube-api-access-sqllj\") pod \"kuadrant-console-plugin-66b5774985-q9ncs\" (UID: \"f72d49b6-73af-48a0-8a20-c84388cfd160\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" Jul 15 17:05:55.167346 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.167140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f72d49b6-73af-48a0-8a20-c84388cfd160-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q9ncs\" (UID: \"f72d49b6-73af-48a0-8a20-c84388cfd160\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" Jul 15 17:05:55.167734 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.167709 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f72d49b6-73af-48a0-8a20-c84388cfd160-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q9ncs\" (UID: \"f72d49b6-73af-48a0-8a20-c84388cfd160\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" Jul 15 17:05:55.169593 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.169570 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f72d49b6-73af-48a0-8a20-c84388cfd160-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q9ncs\" (UID: \"f72d49b6-73af-48a0-8a20-c84388cfd160\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" Jul 15 17:05:55.178468 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.178447 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqllj\" (UniqueName: \"kubernetes.io/projected/f72d49b6-73af-48a0-8a20-c84388cfd160-kube-api-access-sqllj\") pod \"kuadrant-console-plugin-66b5774985-q9ncs\" (UID: \"f72d49b6-73af-48a0-8a20-c84388cfd160\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" Jul 15 17:05:55.227356 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.227319 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" Jul 15 17:05:55.369012 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:55.368984 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs"] Jul 15 17:05:55.370588 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:05:55.370562 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf72d49b6_73af_48a0_8a20_c84388cfd160.slice/crio-1a71be85e56d3ccfa196311600c822c89530de64b6e5771ca49da54ae509c6b1 WatchSource:0}: Error finding container 1a71be85e56d3ccfa196311600c822c89530de64b6e5771ca49da54ae509c6b1: Status 404 returned error can't find the container with id 1a71be85e56d3ccfa196311600c822c89530de64b6e5771ca49da54ae509c6b1 Jul 15 17:05:56.172844 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:56.172808 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" event={"ID":"f72d49b6-73af-48a0-8a20-c84388cfd160","Type":"ContainerStarted","Data":"1a71be85e56d3ccfa196311600c822c89530de64b6e5771ca49da54ae509c6b1"} Jul 15 17:05:56.522005 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:56.521913 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:05:56.623284 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:05:56.623250 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:03.156991 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:03.156960 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vjz6k" Jul 15 17:06:03.157448 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:03.157116 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:06:05.146976 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.146936 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:05.240625 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.240591 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:05.256763 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.256732 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww"] Jul 15 17:06:05.256942 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.256917 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" containerName="manager" containerID="cri-o://442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704" gracePeriod=2 Jul 15 17:06:05.265872 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.265826 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww"] Jul 15 17:06:05.288338 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.288196 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2"] Jul 15 17:06:05.288518 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.288444 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" containerName="manager" containerID="cri-o://4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818" gracePeriod=2 Jul 15 17:06:05.304006 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:05.303872 2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5e8d4224-9795-46ed-a414-2a84f99dc90a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-76dt2\": pods \"limitador-operator-controller-manager-859d7b78bd-76dt2\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:06:05.304521 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.304470 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2"] Jul 15 17:06:05.313068 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:05.313016 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:06:05.324587 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.324480 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4"] Jul 15 17:06:05.325005 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.324985 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" containerName="manager" Jul 15 17:06:05.325005 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.325007 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" containerName="manager" Jul 15 17:06:05.325246 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.325028 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" containerName="manager" Jul 15 17:06:05.325246 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.325038 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" containerName="manager" Jul 15 17:06:05.325246 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.325104 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" containerName="manager" Jul 15 17:06:05.325246 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.325120 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" containerName="manager" Jul 15 17:06:05.329982 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.329962 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4" Jul 15 17:06:05.332653 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:05.332595 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:06:05.335098 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:05.334987 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-76dt2\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:06:05.351863 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.350088 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4"] Jul 15 17:06:05.364704 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.364670 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntkmc\" (UniqueName: \"kubernetes.io/projected/18063618-b182-4e55-a8ac-2d08c7a8f0a4-kube-api-access-ntkmc\") pod \"limitador-operator-controller-manager-859d7b78bd-2rqz4\" (UID: \"18063618-b182-4e55-a8ac-2d08c7a8f0a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4" Jul 15 17:06:05.466095 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.465957 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntkmc\" (UniqueName: \"kubernetes.io/projected/18063618-b182-4e55-a8ac-2d08c7a8f0a4-kube-api-access-ntkmc\") pod \"limitador-operator-controller-manager-859d7b78bd-2rqz4\" (UID: \"18063618-b182-4e55-a8ac-2d08c7a8f0a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4" Jul 15 17:06:05.479847 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.479792 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntkmc\" (UniqueName: \"kubernetes.io/projected/18063618-b182-4e55-a8ac-2d08c7a8f0a4-kube-api-access-ntkmc\") pod \"limitador-operator-controller-manager-859d7b78bd-2rqz4\" (UID: \"18063618-b182-4e55-a8ac-2d08c7a8f0a4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4" Jul 15 17:06:05.685335 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.685298 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4" Jul 15 17:06:05.715670 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.715638 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh"] Jul 15 17:06:05.719347 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.719292 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:06:05.742974 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.742950 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh"] Jul 15 17:06:05.768618 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.768587 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11ec1563-96bb-4300-b600-8693ef3738ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4ppqh\" (UID: \"11ec1563-96bb-4300-b600-8693ef3738ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:06:05.768784 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.768639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqs9w\" (UniqueName: \"kubernetes.io/projected/11ec1563-96bb-4300-b600-8693ef3738ad-kube-api-access-sqs9w\") pod \"kuadrant-operator-controller-manager-86878c769f-4ppqh\" (UID: \"11ec1563-96bb-4300-b600-8693ef3738ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:06:05.778338 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:05.778303 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-76dt2\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:06:05.781728 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:05.781696 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:06:05.869994 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.869960 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11ec1563-96bb-4300-b600-8693ef3738ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4ppqh\" (UID: \"11ec1563-96bb-4300-b600-8693ef3738ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:06:05.870230 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.870018 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqs9w\" (UniqueName: \"kubernetes.io/projected/11ec1563-96bb-4300-b600-8693ef3738ad-kube-api-access-sqs9w\") pod \"kuadrant-operator-controller-manager-86878c769f-4ppqh\" (UID: \"11ec1563-96bb-4300-b600-8693ef3738ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:06:05.870386 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.870365 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11ec1563-96bb-4300-b600-8693ef3738ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4ppqh\" (UID: \"11ec1563-96bb-4300-b600-8693ef3738ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:06:05.888353 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:05.888329 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqs9w\" (UniqueName: \"kubernetes.io/projected/11ec1563-96bb-4300-b600-8693ef3738ad-kube-api-access-sqs9w\") pod \"kuadrant-operator-controller-manager-86878c769f-4ppqh\" (UID: \"11ec1563-96bb-4300-b600-8693ef3738ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:06:06.032483 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:06.032405 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:06:07.649100 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:07.649067 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:07.788286 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:07.788245 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:11.932160 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:11.932115 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:12.063175 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:12.063139 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:14.166473 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:14.166439 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:14.269109 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:14.269070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:18.605385 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.605355 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4"] Jul 15 17:06:18.607022 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:06:18.606991 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18063618_b182_4e55_a8ac_2d08c7a8f0a4.slice/crio-5d9c043b60bbe2fc508d5345f59bb4fe41d8ea3c70c30ba7d227945e0221da8a WatchSource:0}: Error finding container 5d9c043b60bbe2fc508d5345f59bb4fe41d8ea3c70c30ba7d227945e0221da8a: Status 404 returned error can't find the container with id 5d9c043b60bbe2fc508d5345f59bb4fe41d8ea3c70c30ba7d227945e0221da8a Jul 15 17:06:18.626226 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.626192 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh"] Jul 15 17:06:18.806622 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:06:18.806582 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ec1563_96bb_4300_b600_8693ef3738ad.slice/crio-9b00dc06f6e13e6263326b1a18920ac064f68ea0347d6380205393acc15c80e2 WatchSource:0}: Error finding container 9b00dc06f6e13e6263326b1a18920ac064f68ea0347d6380205393acc15c80e2: Status 404 returned error can't find the container with id 9b00dc06f6e13e6263326b1a18920ac064f68ea0347d6380205393acc15c80e2 Jul 15 17:06:18.827772 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.827749 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:06:18.830133 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:18.830102 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-76dt2\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:06:18.844170 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.844148 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:06:18.846740 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:18.846716 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-76dt2\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:06:18.848919 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:18.848894 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:06:18.887139 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.887117 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwwvn\" (UniqueName: \"kubernetes.io/projected/5e8d4224-9795-46ed-a414-2a84f99dc90a-kube-api-access-dwwvn\") pod \"5e8d4224-9795-46ed-a414-2a84f99dc90a\" (UID: \"5e8d4224-9795-46ed-a414-2a84f99dc90a\") " Jul 15 17:06:18.887239 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.887170 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcf8x\" (UniqueName: \"kubernetes.io/projected/2639457a-bba8-4fdd-a7d6-2759454c5235-kube-api-access-vcf8x\") pod \"2639457a-bba8-4fdd-a7d6-2759454c5235\" (UID: \"2639457a-bba8-4fdd-a7d6-2759454c5235\") " Jul 15 17:06:18.887239 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.887197 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2639457a-bba8-4fdd-a7d6-2759454c5235-extensions-socket-volume\") pod \"2639457a-bba8-4fdd-a7d6-2759454c5235\" (UID: \"2639457a-bba8-4fdd-a7d6-2759454c5235\") " Jul 15 17:06:18.887727 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.887707 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2639457a-bba8-4fdd-a7d6-2759454c5235-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2639457a-bba8-4fdd-a7d6-2759454c5235" (UID: "2639457a-bba8-4fdd-a7d6-2759454c5235"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:06:18.888993 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.888971 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e8d4224-9795-46ed-a414-2a84f99dc90a-kube-api-access-dwwvn" (OuterVolumeSpecName: "kube-api-access-dwwvn") pod "5e8d4224-9795-46ed-a414-2a84f99dc90a" (UID: "5e8d4224-9795-46ed-a414-2a84f99dc90a"). InnerVolumeSpecName "kube-api-access-dwwvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:06:18.889208 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.889187 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2639457a-bba8-4fdd-a7d6-2759454c5235-kube-api-access-vcf8x" (OuterVolumeSpecName: "kube-api-access-vcf8x") pod "2639457a-bba8-4fdd-a7d6-2759454c5235" (UID: "2639457a-bba8-4fdd-a7d6-2759454c5235"). InnerVolumeSpecName "kube-api-access-vcf8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:06:18.987696 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.987667 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwwvn\" (UniqueName: \"kubernetes.io/projected/5e8d4224-9795-46ed-a414-2a84f99dc90a-kube-api-access-dwwvn\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:06:18.987696 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.987689 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcf8x\" (UniqueName: \"kubernetes.io/projected/2639457a-bba8-4fdd-a7d6-2759454c5235-kube-api-access-vcf8x\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:06:18.987696 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:18.987699 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2639457a-bba8-4fdd-a7d6-2759454c5235-extensions-socket-volume\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:06:19.318349 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.318315 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" event={"ID":"f72d49b6-73af-48a0-8a20-c84388cfd160","Type":"ContainerStarted","Data":"eb062252a5f48467d60d09ec32192ab73a0176e3b5dbcf5f43666fba88677ca3"} Jul 15 17:06:19.320061 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.320026 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4" event={"ID":"18063618-b182-4e55-a8ac-2d08c7a8f0a4","Type":"ContainerStarted","Data":"ce7c43769ee7ae710945395690827d95c31d78694afb7177682c0684aa77c2c5"} Jul 15 17:06:19.320189 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.320086 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4" Jul 15 17:06:19.320189 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.320112 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4" event={"ID":"18063618-b182-4e55-a8ac-2d08c7a8f0a4","Type":"ContainerStarted","Data":"5d9c043b60bbe2fc508d5345f59bb4fe41d8ea3c70c30ba7d227945e0221da8a"} Jul 15 17:06:19.321274 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.321253 2562 generic.go:356] "Generic (PLEG): container finished" podID="5e8d4224-9795-46ed-a414-2a84f99dc90a" containerID="4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818" exitCode=0 Jul 15 17:06:19.321373 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.321289 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:06:19.321373 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.321339 2562 scope.go:117] "RemoveContainer" containerID="4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818" Jul 15 17:06:19.323032 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.322954 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" event={"ID":"11ec1563-96bb-4300-b600-8693ef3738ad","Type":"ContainerStarted","Data":"d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def"} Jul 15 17:06:19.323032 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.323005 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:06:19.323032 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.323027 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" event={"ID":"11ec1563-96bb-4300-b600-8693ef3738ad","Type":"ContainerStarted","Data":"9b00dc06f6e13e6263326b1a18920ac064f68ea0347d6380205393acc15c80e2"} Jul 15 17:06:19.324397 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.324372 2562 generic.go:356] "Generic (PLEG): container finished" podID="2639457a-bba8-4fdd-a7d6-2759454c5235" containerID="442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704" exitCode=0 Jul 15 17:06:19.324397 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.324396 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:06:19.332384 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.332368 2562 scope.go:117] "RemoveContainer" containerID="4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818" Jul 15 17:06:19.332651 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:19.332629 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818\": container with ID starting with 4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818 not found: ID does not exist" containerID="4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818" Jul 15 17:06:19.332724 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.332664 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818"} err="failed to get container status \"4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818\": rpc error: code = NotFound desc = could not find container \"4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818\": container with ID starting with 4ab2e4ed8af475993dbec52bda7e7f303ad08e7f1e8efe4f4a8c5750ae5bc818 not found: ID does not exist" Jul 15 17:06:19.332724 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.332687 2562 scope.go:117] "RemoveContainer" containerID="442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704" Jul 15 17:06:19.339788 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.339743 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q9ncs" podStartSLOduration=1.840210605 podStartE2EDuration="25.339730283s" totalImagesPullingTime="23.499519678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:19.337990331 +0000 UTC m=+429.579985569" watchObservedRunningTime="2026-07-15 17:06:19.339730283 +0000 UTC m=+429.581725508" Jul 15 17:06:19.340492 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:19.340468 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-76dt2\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:06:19.341614 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.341598 2562 scope.go:117] "RemoveContainer" containerID="442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704" Jul 15 17:06:19.341877 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:19.341862 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704\": container with ID starting with 442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704 not found: ID does not exist" containerID="442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704" Jul 15 17:06:19.341925 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.341884 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704"} err="failed to get container status \"442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704\": rpc error: code = NotFound desc = could not find container \"442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704\": container with ID starting with 442d88533def9a7f770f1899621695aff435c9a27fc8be1ac65a236d4db20704 not found: ID does not exist" Jul 15 17:06:19.342523 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:19.342506 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:06:19.367573 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:19.367537 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-76dt2\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:06:19.368672 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.368629 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" podStartSLOduration=14.36861322 podStartE2EDuration="14.36861322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:19.365049077 +0000 UTC m=+429.607044304" watchObservedRunningTime="2026-07-15 17:06:19.36861322 +0000 UTC m=+429.610608448" Jul 15 17:06:19.388698 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:19.388653 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4" podStartSLOduration=14.388636906 podStartE2EDuration="14.388636906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:19.386812859 +0000 UTC m=+429.628808085" watchObservedRunningTime="2026-07-15 17:06:19.388636906 +0000 UTC m=+429.630632133" Jul 15 17:06:19.389197 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:19.389172 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:06:20.250120 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:20.250088 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-76dt2\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-76dt2" Jul 15 17:06:20.250679 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:20.250660 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" path="/var/lib/kubelet/pods/2639457a-bba8-4fdd-a7d6-2759454c5235/volumes" Jul 15 17:06:20.250997 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:20.250983 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e8d4224-9795-46ed-a414-2a84f99dc90a" path="/var/lib/kubelet/pods/5e8d4224-9795-46ed-a414-2a84f99dc90a/volumes" Jul 15 17:06:20.252182 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:20.252158 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r2nww\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="2639457a-bba8-4fdd-a7d6-2759454c5235" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2nww" Jul 15 17:06:22.945789 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:22.945744 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:23.136586 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:23.136554 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:25.517340 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:25.517302 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:25.658000 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:25.657971 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:29.877396 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:29.877362 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:30.014219 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:30.014183 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:30.333683 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:30.333657 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:06:30.333830 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:30.333727 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rqz4" Jul 15 17:06:30.944378 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:30.944295 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:31.055032 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:31.055000 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:32.539704 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:32.539667 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:32.637202 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:32.637165 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:42.511185 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:42.511146 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:42.575246 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:42.575209 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:42.935491 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:42.935462 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:43.015862 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:43.015802 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:45.954130 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:45.954083 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:46.129715 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:46.129680 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:46.979017 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:46.978977 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:50.155783 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:50.155734 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:50.248572 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:50.248538 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:50.688348 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:50.688317 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-gwpbt"] Jul 15 17:06:50.736492 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:50.736462 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gwpbt"] Jul 15 17:06:50.736649 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:50.736555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gwpbt" Jul 15 17:06:50.745608 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:50.745580 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:50.747742 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:50.747722 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pwpzs\"" Jul 15 17:06:50.750181 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:50.750158 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v99s\" (UniqueName: \"kubernetes.io/projected/16c10286-36a5-438d-b7f6-ebc7982330d0-kube-api-access-6v99s\") pod \"authorino-58cd955675-gwpbt\" (UID: \"16c10286-36a5-438d-b7f6-ebc7982330d0\") " pod="kuadrant-system/authorino-58cd955675-gwpbt" Jul 15 17:06:50.850748 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:50.850719 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v99s\" (UniqueName: \"kubernetes.io/projected/16c10286-36a5-438d-b7f6-ebc7982330d0-kube-api-access-6v99s\") pod \"authorino-58cd955675-gwpbt\" (UID: \"16c10286-36a5-438d-b7f6-ebc7982330d0\") " pod="kuadrant-system/authorino-58cd955675-gwpbt" Jul 15 17:06:50.859385 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:50.859349 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:50.863301 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:50.863281 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v99s\" (UniqueName: \"kubernetes.io/projected/16c10286-36a5-438d-b7f6-ebc7982330d0-kube-api-access-6v99s\") pod \"authorino-58cd955675-gwpbt\" (UID: \"16c10286-36a5-438d-b7f6-ebc7982330d0\") " pod="kuadrant-system/authorino-58cd955675-gwpbt" Jul 15 17:06:51.046962 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:51.046888 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gwpbt" Jul 15 17:06:51.195926 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:51.195892 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gwpbt"] Jul 15 17:06:51.199291 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:06:51.199266 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16c10286_36a5_438d_b7f6_ebc7982330d0.slice/crio-184b267dba30ce5a090394b0bf34a556c049e90b3b4fb29a05bcfe7aa6067919 WatchSource:0}: Error finding container 184b267dba30ce5a090394b0bf34a556c049e90b3b4fb29a05bcfe7aa6067919: Status 404 returned error can't find the container with id 184b267dba30ce5a090394b0bf34a556c049e90b3b4fb29a05bcfe7aa6067919 Jul 15 17:06:51.494394 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:51.494360 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gwpbt" event={"ID":"16c10286-36a5-438d-b7f6-ebc7982330d0","Type":"ContainerStarted","Data":"184b267dba30ce5a090394b0bf34a556c049e90b3b4fb29a05bcfe7aa6067919"} Jul 15 17:06:56.325038 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:56.325006 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gwpbt"] Jul 15 17:06:56.421690 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:56.421661 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:56.521747 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:56.521718 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:06:57.535492 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:57.535459 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gwpbt" event={"ID":"16c10286-36a5-438d-b7f6-ebc7982330d0","Type":"ContainerStarted","Data":"b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f"} Jul 15 17:06:57.535951 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:57.535587 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-gwpbt" podUID="16c10286-36a5-438d-b7f6-ebc7982330d0" containerName="authorino" containerID="cri-o://b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f" gracePeriod=30 Jul 15 17:06:57.560143 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:57.560084 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-gwpbt" podStartSLOduration=1.351320187 podStartE2EDuration="7.560064722s" totalImagesPullingTime="6.208744535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:57.550974077 +0000 UTC m=+467.792969315" watchObservedRunningTime="2026-07-15 17:06:57.560064722 +0000 UTC m=+467.802059950" Jul 15 17:06:57.777725 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:57.777699 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gwpbt" Jul 15 17:06:57.808180 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:57.808156 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v99s\" (UniqueName: \"kubernetes.io/projected/16c10286-36a5-438d-b7f6-ebc7982330d0-kube-api-access-6v99s\") pod \"16c10286-36a5-438d-b7f6-ebc7982330d0\" (UID: \"16c10286-36a5-438d-b7f6-ebc7982330d0\") " Jul 15 17:06:57.810200 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:57.810179 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16c10286-36a5-438d-b7f6-ebc7982330d0-kube-api-access-6v99s" (OuterVolumeSpecName: "kube-api-access-6v99s") pod "16c10286-36a5-438d-b7f6-ebc7982330d0" (UID: "16c10286-36a5-438d-b7f6-ebc7982330d0"). InnerVolumeSpecName "kube-api-access-6v99s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:06:57.908806 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:57.908783 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v99s\" (UniqueName: \"kubernetes.io/projected/16c10286-36a5-438d-b7f6-ebc7982330d0-kube-api-access-6v99s\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:06:58.542890 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:58.542777 2562 generic.go:356] "Generic (PLEG): container finished" podID="16c10286-36a5-438d-b7f6-ebc7982330d0" containerID="b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f" exitCode=0 Jul 15 17:06:58.542890 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:58.542821 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gwpbt" Jul 15 17:06:58.542890 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:58.542826 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gwpbt" event={"ID":"16c10286-36a5-438d-b7f6-ebc7982330d0","Type":"ContainerDied","Data":"b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f"} Jul 15 17:06:58.542890 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:58.542891 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gwpbt" event={"ID":"16c10286-36a5-438d-b7f6-ebc7982330d0","Type":"ContainerDied","Data":"184b267dba30ce5a090394b0bf34a556c049e90b3b4fb29a05bcfe7aa6067919"} Jul 15 17:06:58.543671 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:58.542909 2562 scope.go:117] "RemoveContainer" containerID="b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f" Jul 15 17:06:58.551518 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:58.551500 2562 scope.go:117] "RemoveContainer" containerID="b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f" Jul 15 17:06:58.551782 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:06:58.551763 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f\": container with ID starting with b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f not found: ID does not exist" containerID="b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f" Jul 15 17:06:58.551829 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:58.551789 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f"} err="failed to get container status \"b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f\": rpc error: code = NotFound desc = could not find container \"b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f\": container with ID starting with b9369baf2143549ab4ac48221f47936bc76630685b56137a67c76236af157f3f not found: ID does not exist" Jul 15 17:06:58.562133 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:58.562113 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gwpbt"] Jul 15 17:06:58.569493 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:06:58.569473 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-gwpbt"] Jul 15 17:07:00.250763 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:00.250735 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16c10286-36a5-438d-b7f6-ebc7982330d0" path="/var/lib/kubelet/pods/16c10286-36a5-438d-b7f6-ebc7982330d0/volumes" Jul 15 17:07:10.070312 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:10.070280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:10.298900 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:10.298859 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:10.585810 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:10.585775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:22.722852 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:22.722794 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:22.822831 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:22.822776 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:22.957750 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:22.957719 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4768s"] Jul 15 17:07:22.958152 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:22.958138 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16c10286-36a5-438d-b7f6-ebc7982330d0" containerName="authorino" Jul 15 17:07:22.958203 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:22.958154 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="16c10286-36a5-438d-b7f6-ebc7982330d0" containerName="authorino" Jul 15 17:07:22.958237 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:22.958220 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="16c10286-36a5-438d-b7f6-ebc7982330d0" containerName="authorino" Jul 15 17:07:22.962644 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:22.962614 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4768s" Jul 15 17:07:22.965381 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:22.965358 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pwpzs\"" Jul 15 17:07:22.968046 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:22.968023 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4768s"] Jul 15 17:07:23.018262 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.018191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfpmd\" (UniqueName: \"kubernetes.io/projected/139c69be-6dab-40a7-872b-e20cfbbf2999-kube-api-access-sfpmd\") pod \"authorino-5d4b67c446-4768s\" (UID: \"139c69be-6dab-40a7-872b-e20cfbbf2999\") " pod="kuadrant-system/authorino-5d4b67c446-4768s" Jul 15 17:07:23.119374 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.119341 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfpmd\" (UniqueName: \"kubernetes.io/projected/139c69be-6dab-40a7-872b-e20cfbbf2999-kube-api-access-sfpmd\") pod \"authorino-5d4b67c446-4768s\" (UID: \"139c69be-6dab-40a7-872b-e20cfbbf2999\") " pod="kuadrant-system/authorino-5d4b67c446-4768s" Jul 15 17:07:23.128281 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.128259 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfpmd\" (UniqueName: \"kubernetes.io/projected/139c69be-6dab-40a7-872b-e20cfbbf2999-kube-api-access-sfpmd\") pod \"authorino-5d4b67c446-4768s\" (UID: \"139c69be-6dab-40a7-872b-e20cfbbf2999\") " pod="kuadrant-system/authorino-5d4b67c446-4768s" Jul 15 17:07:23.208925 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.208898 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4768s"] Jul 15 17:07:23.209117 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.209104 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4768s" Jul 15 17:07:23.262802 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.262774 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69fd956586-n4w68"] Jul 15 17:07:23.267369 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.267350 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69fd956586-n4w68" Jul 15 17:07:23.283233 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.283210 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69fd956586-n4w68"] Jul 15 17:07:23.321297 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.321266 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t498b\" (UniqueName: \"kubernetes.io/projected/1fc9005b-ab70-4dc8-9a0d-c52cf21d0079-kube-api-access-t498b\") pod \"authorino-69fd956586-n4w68\" (UID: \"1fc9005b-ab70-4dc8-9a0d-c52cf21d0079\") " pod="kuadrant-system/authorino-69fd956586-n4w68" Jul 15 17:07:23.366292 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.366271 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4768s"] Jul 15 17:07:23.368672 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:07:23.368641 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod139c69be_6dab_40a7_872b_e20cfbbf2999.slice/crio-e5a1ff0aa6adaecda7a92507374e527c6252f9a6b5d91fb7b93779ff956b1842 WatchSource:0}: Error finding container e5a1ff0aa6adaecda7a92507374e527c6252f9a6b5d91fb7b93779ff956b1842: Status 404 returned error can't find the container with id e5a1ff0aa6adaecda7a92507374e527c6252f9a6b5d91fb7b93779ff956b1842 Jul 15 17:07:23.421793 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.421766 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t498b\" (UniqueName: \"kubernetes.io/projected/1fc9005b-ab70-4dc8-9a0d-c52cf21d0079-kube-api-access-t498b\") pod \"authorino-69fd956586-n4w68\" (UID: \"1fc9005b-ab70-4dc8-9a0d-c52cf21d0079\") " pod="kuadrant-system/authorino-69fd956586-n4w68" Jul 15 17:07:23.436080 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.436058 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t498b\" (UniqueName: \"kubernetes.io/projected/1fc9005b-ab70-4dc8-9a0d-c52cf21d0079-kube-api-access-t498b\") pod \"authorino-69fd956586-n4w68\" (UID: \"1fc9005b-ab70-4dc8-9a0d-c52cf21d0079\") " pod="kuadrant-system/authorino-69fd956586-n4w68" Jul 15 17:07:23.447212 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.447171 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69fd956586-n4w68"] Jul 15 17:07:23.447436 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.447425 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69fd956586-n4w68" Jul 15 17:07:23.587386 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.587360 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69fd956586-n4w68"] Jul 15 17:07:23.589226 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:07:23.589198 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fc9005b_ab70_4dc8_9a0d_c52cf21d0079.slice/crio-13cb4a016116cc40eaf27598a97b6d7ba861e72360f8b9ecc24d7ea202039b19 WatchSource:0}: Error finding container 13cb4a016116cc40eaf27598a97b6d7ba861e72360f8b9ecc24d7ea202039b19: Status 404 returned error can't find the container with id 13cb4a016116cc40eaf27598a97b6d7ba861e72360f8b9ecc24d7ea202039b19 Jul 15 17:07:23.675063 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.675032 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4768s" event={"ID":"139c69be-6dab-40a7-872b-e20cfbbf2999","Type":"ContainerStarted","Data":"e5a1ff0aa6adaecda7a92507374e527c6252f9a6b5d91fb7b93779ff956b1842"} Jul 15 17:07:23.676202 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:23.676180 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69fd956586-n4w68" event={"ID":"1fc9005b-ab70-4dc8-9a0d-c52cf21d0079","Type":"ContainerStarted","Data":"13cb4a016116cc40eaf27598a97b6d7ba861e72360f8b9ecc24d7ea202039b19"} Jul 15 17:07:24.682437 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:24.682402 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69fd956586-n4w68" event={"ID":"1fc9005b-ab70-4dc8-9a0d-c52cf21d0079","Type":"ContainerStarted","Data":"02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0"} Jul 15 17:07:24.682909 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:24.682401 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-69fd956586-n4w68" podUID="1fc9005b-ab70-4dc8-9a0d-c52cf21d0079" containerName="authorino" containerID="cri-o://02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0" gracePeriod=30 Jul 15 17:07:24.684046 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:24.684026 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4768s" event={"ID":"139c69be-6dab-40a7-872b-e20cfbbf2999","Type":"ContainerStarted","Data":"43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661"} Jul 15 17:07:24.684149 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:24.684123 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-4768s" podUID="139c69be-6dab-40a7-872b-e20cfbbf2999" containerName="authorino" containerID="cri-o://43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661" gracePeriod=30 Jul 15 17:07:24.699944 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:24.699889 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69fd956586-n4w68" podStartSLOduration=1.3250957620000001 podStartE2EDuration="1.699872302s" totalImagesPullingTime="374.77654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:07:24.699349596 +0000 UTC m=+494.941344825" watchObservedRunningTime="2026-07-15 17:07:24.699872302 +0000 UTC m=+494.941867529" Jul 15 17:07:24.718571 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:24.718529 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-4768s" podStartSLOduration=2.303975085 podStartE2EDuration="2.718515771s" totalImagesPullingTime="414.540686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:07:24.716406082 +0000 UTC m=+494.958401308" watchObservedRunningTime="2026-07-15 17:07:24.718515771 +0000 UTC m=+494.960510997" Jul 15 17:07:24.954220 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:24.954194 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69fd956586-n4w68" Jul 15 17:07:24.956924 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:24.956910 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4768s" Jul 15 17:07:25.035401 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.035366 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t498b\" (UniqueName: \"kubernetes.io/projected/1fc9005b-ab70-4dc8-9a0d-c52cf21d0079-kube-api-access-t498b\") pod \"1fc9005b-ab70-4dc8-9a0d-c52cf21d0079\" (UID: \"1fc9005b-ab70-4dc8-9a0d-c52cf21d0079\") " Jul 15 17:07:25.035565 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.035411 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfpmd\" (UniqueName: \"kubernetes.io/projected/139c69be-6dab-40a7-872b-e20cfbbf2999-kube-api-access-sfpmd\") pod \"139c69be-6dab-40a7-872b-e20cfbbf2999\" (UID: \"139c69be-6dab-40a7-872b-e20cfbbf2999\") " Jul 15 17:07:25.037666 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.037640 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/139c69be-6dab-40a7-872b-e20cfbbf2999-kube-api-access-sfpmd" (OuterVolumeSpecName: "kube-api-access-sfpmd") pod "139c69be-6dab-40a7-872b-e20cfbbf2999" (UID: "139c69be-6dab-40a7-872b-e20cfbbf2999"). InnerVolumeSpecName "kube-api-access-sfpmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:07:25.037723 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.037683 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fc9005b-ab70-4dc8-9a0d-c52cf21d0079-kube-api-access-t498b" (OuterVolumeSpecName: "kube-api-access-t498b") pod "1fc9005b-ab70-4dc8-9a0d-c52cf21d0079" (UID: "1fc9005b-ab70-4dc8-9a0d-c52cf21d0079"). InnerVolumeSpecName "kube-api-access-t498b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:07:25.136361 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.136340 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t498b\" (UniqueName: \"kubernetes.io/projected/1fc9005b-ab70-4dc8-9a0d-c52cf21d0079-kube-api-access-t498b\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:07:25.136451 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.136364 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfpmd\" (UniqueName: \"kubernetes.io/projected/139c69be-6dab-40a7-872b-e20cfbbf2999-kube-api-access-sfpmd\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:07:25.690508 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.690472 2562 generic.go:356] "Generic (PLEG): container finished" podID="139c69be-6dab-40a7-872b-e20cfbbf2999" containerID="43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661" exitCode=0 Jul 15 17:07:25.690972 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.690528 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4768s" Jul 15 17:07:25.690972 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.690552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4768s" event={"ID":"139c69be-6dab-40a7-872b-e20cfbbf2999","Type":"ContainerDied","Data":"43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661"} Jul 15 17:07:25.690972 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.690591 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4768s" event={"ID":"139c69be-6dab-40a7-872b-e20cfbbf2999","Type":"ContainerDied","Data":"e5a1ff0aa6adaecda7a92507374e527c6252f9a6b5d91fb7b93779ff956b1842"} Jul 15 17:07:25.690972 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.690609 2562 scope.go:117] "RemoveContainer" containerID="43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661" Jul 15 17:07:25.692200 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.692132 2562 generic.go:356] "Generic (PLEG): container finished" podID="1fc9005b-ab70-4dc8-9a0d-c52cf21d0079" containerID="02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0" exitCode=0 Jul 15 17:07:25.692290 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.692162 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69fd956586-n4w68" event={"ID":"1fc9005b-ab70-4dc8-9a0d-c52cf21d0079","Type":"ContainerDied","Data":"02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0"} Jul 15 17:07:25.692290 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.692283 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69fd956586-n4w68" event={"ID":"1fc9005b-ab70-4dc8-9a0d-c52cf21d0079","Type":"ContainerDied","Data":"13cb4a016116cc40eaf27598a97b6d7ba861e72360f8b9ecc24d7ea202039b19"} Jul 15 17:07:25.692370 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.692171 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69fd956586-n4w68" Jul 15 17:07:25.700711 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.700691 2562 scope.go:117] "RemoveContainer" containerID="43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661" Jul 15 17:07:25.701057 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:07:25.701038 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661\": container with ID starting with 43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661 not found: ID does not exist" containerID="43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661" Jul 15 17:07:25.701154 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.701062 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661"} err="failed to get container status \"43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661\": rpc error: code = NotFound desc = could not find container \"43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661\": container with ID starting with 43575f71ba38ca5a1df4d21aed64e97dc3df7eac304a8240580cd0a7af8cf661 not found: ID does not exist" Jul 15 17:07:25.701154 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.701080 2562 scope.go:117] "RemoveContainer" containerID="02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0" Jul 15 17:07:25.708974 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.708955 2562 scope.go:117] "RemoveContainer" containerID="02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0" Jul 15 17:07:25.709211 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:07:25.709192 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0\": container with ID starting with 02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0 not found: ID does not exist" containerID="02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0" Jul 15 17:07:25.709271 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.709220 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0"} err="failed to get container status \"02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0\": rpc error: code = NotFound desc = could not find container \"02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0\": container with ID starting with 02564faae0b92ce23bdc18b18db37c44f6b8bb2cdc7e15e073304b7e9b792ab0 not found: ID does not exist" Jul 15 17:07:25.713569 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.713540 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4768s"] Jul 15 17:07:25.720858 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.718272 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4768s"] Jul 15 17:07:25.737538 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.737502 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69fd956586-n4w68"] Jul 15 17:07:25.744726 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:25.744705 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-69fd956586-n4w68"] Jul 15 17:07:26.252800 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:26.252751 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="139c69be-6dab-40a7-872b-e20cfbbf2999" path="/var/lib/kubelet/pods/139c69be-6dab-40a7-872b-e20cfbbf2999/volumes" Jul 15 17:07:26.253210 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:26.253191 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fc9005b-ab70-4dc8-9a0d-c52cf21d0079" path="/var/lib/kubelet/pods/1fc9005b-ab70-4dc8-9a0d-c52cf21d0079/volumes" Jul 15 17:07:30.184653 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.184620 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bb598f85c-ztwgd"] Jul 15 17:07:30.185155 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.184979 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="139c69be-6dab-40a7-872b-e20cfbbf2999" containerName="authorino" Jul 15 17:07:30.185155 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.184990 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="139c69be-6dab-40a7-872b-e20cfbbf2999" containerName="authorino" Jul 15 17:07:30.185155 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.185006 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc9005b-ab70-4dc8-9a0d-c52cf21d0079" containerName="authorino" Jul 15 17:07:30.185155 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.185011 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc9005b-ab70-4dc8-9a0d-c52cf21d0079" containerName="authorino" Jul 15 17:07:30.185155 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.185084 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="139c69be-6dab-40a7-872b-e20cfbbf2999" containerName="authorino" Jul 15 17:07:30.185155 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.185092 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fc9005b-ab70-4dc8-9a0d-c52cf21d0079" containerName="authorino" Jul 15 17:07:30.191346 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.191326 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:30.196405 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:07:30.196354 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-controller-dockercfg-d6x2t\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d6x2t\"" type="*v1.Secret" Jul 15 17:07:30.196405 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:07:30.196386 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-controller-7bb598f85c-ztwgd\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" podUID="8d6d567e-7555-40be-a78f-cf0b841d5c60" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:30.196587 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:07:30.196567 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" type="*v1.Secret" Jul 15 17:07:30.196889 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:07:30.196872 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"maas-parameters\" is forbidden: User \"system:node:ip-10-0-135-132.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-132.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"maas-parameters\"" type="*v1.ConfigMap" Jul 15 17:07:30.212392 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.212364 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bb598f85c-ztwgd"] Jul 15 17:07:30.275919 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.275886 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pd4j\" (UniqueName: \"kubernetes.io/projected/8d6d567e-7555-40be-a78f-cf0b841d5c60-kube-api-access-8pd4j\") pod \"maas-controller-7bb598f85c-ztwgd\" (UID: \"8d6d567e-7555-40be-a78f-cf0b841d5c60\") " pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:30.276071 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.275937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d6d567e-7555-40be-a78f-cf0b841d5c60-webhook-cert\") pod \"maas-controller-7bb598f85c-ztwgd\" (UID: \"8d6d567e-7555-40be-a78f-cf0b841d5c60\") " pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:30.377695 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.377659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pd4j\" (UniqueName: \"kubernetes.io/projected/8d6d567e-7555-40be-a78f-cf0b841d5c60-kube-api-access-8pd4j\") pod \"maas-controller-7bb598f85c-ztwgd\" (UID: \"8d6d567e-7555-40be-a78f-cf0b841d5c60\") " pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:30.377908 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.377743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d6d567e-7555-40be-a78f-cf0b841d5c60-webhook-cert\") pod \"maas-controller-7bb598f85c-ztwgd\" (UID: \"8d6d567e-7555-40be-a78f-cf0b841d5c60\") " pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:30.388768 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.388740 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pd4j\" (UniqueName: \"kubernetes.io/projected/8d6d567e-7555-40be-a78f-cf0b841d5c60-kube-api-access-8pd4j\") pod \"maas-controller-7bb598f85c-ztwgd\" (UID: \"8d6d567e-7555-40be-a78f-cf0b841d5c60\") " pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:30.402851 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.402807 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6964775bf7-jdsds"] Jul 15 17:07:30.406268 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.406248 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:07:30.420940 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.420917 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6964775bf7-jdsds"] Jul 15 17:07:30.479230 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.479156 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q2wk\" (UniqueName: \"kubernetes.io/projected/94cb3a78-9625-40e8-962a-20805e23eac2-kube-api-access-2q2wk\") pod \"maas-controller-6964775bf7-jdsds\" (UID: \"94cb3a78-9625-40e8-962a-20805e23eac2\") " pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:07:30.479230 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.479204 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94cb3a78-9625-40e8-962a-20805e23eac2-webhook-cert\") pod \"maas-controller-6964775bf7-jdsds\" (UID: \"94cb3a78-9625-40e8-962a-20805e23eac2\") " pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:07:30.579620 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.579590 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q2wk\" (UniqueName: \"kubernetes.io/projected/94cb3a78-9625-40e8-962a-20805e23eac2-kube-api-access-2q2wk\") pod \"maas-controller-6964775bf7-jdsds\" (UID: \"94cb3a78-9625-40e8-962a-20805e23eac2\") " pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:07:30.579762 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.579659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94cb3a78-9625-40e8-962a-20805e23eac2-webhook-cert\") pod \"maas-controller-6964775bf7-jdsds\" (UID: \"94cb3a78-9625-40e8-962a-20805e23eac2\") " pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:07:30.589240 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:30.589214 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q2wk\" (UniqueName: \"kubernetes.io/projected/94cb3a78-9625-40e8-962a-20805e23eac2-kube-api-access-2q2wk\") pod \"maas-controller-6964775bf7-jdsds\" (UID: \"94cb3a78-9625-40e8-962a-20805e23eac2\") " pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:07:31.379145 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:07:31.379092 2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: failed to sync secret cache: timed out waiting for the condition Jul 15 17:07:31.379548 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:07:31.379192 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d6d567e-7555-40be-a78f-cf0b841d5c60-webhook-cert podName:8d6d567e-7555-40be-a78f-cf0b841d5c60 nodeName:}" failed. No retries permitted until 2026-07-15 17:07:31.879174089 +0000 UTC m=+502.121169292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/8d6d567e-7555-40be-a78f-cf0b841d5c60-webhook-cert") pod "maas-controller-7bb598f85c-ztwgd" (UID: "8d6d567e-7555-40be-a78f-cf0b841d5c60") : failed to sync secret cache: timed out waiting for the condition Jul 15 17:07:31.546343 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:31.546311 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 17:07:31.580924 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:07:31.580893 2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: failed to sync secret cache: timed out waiting for the condition Jul 15 17:07:31.581084 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:07:31.580972 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94cb3a78-9625-40e8-962a-20805e23eac2-webhook-cert podName:94cb3a78-9625-40e8-962a-20805e23eac2 nodeName:}" failed. No retries permitted until 2026-07-15 17:07:32.080957361 +0000 UTC m=+502.322952564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/94cb3a78-9625-40e8-962a-20805e23eac2-webhook-cert") pod "maas-controller-6964775bf7-jdsds" (UID: "94cb3a78-9625-40e8-962a-20805e23eac2") : failed to sync secret cache: timed out waiting for the condition Jul 15 17:07:31.678719 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:31.678633 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 17:07:31.766039 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:31.765997 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d6x2t\"" Jul 15 17:07:31.890527 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:31.890482 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d6d567e-7555-40be-a78f-cf0b841d5c60-webhook-cert\") pod \"maas-controller-7bb598f85c-ztwgd\" (UID: \"8d6d567e-7555-40be-a78f-cf0b841d5c60\") " pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:31.893134 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:31.893112 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d6d567e-7555-40be-a78f-cf0b841d5c60-webhook-cert\") pod \"maas-controller-7bb598f85c-ztwgd\" (UID: \"8d6d567e-7555-40be-a78f-cf0b841d5c60\") " pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:31.893816 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:31.893789 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:32.000889 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:32.000778 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:32.092658 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:32.092624 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94cb3a78-9625-40e8-962a-20805e23eac2-webhook-cert\") pod \"maas-controller-6964775bf7-jdsds\" (UID: \"94cb3a78-9625-40e8-962a-20805e23eac2\") " pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:07:32.095198 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:32.095177 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94cb3a78-9625-40e8-962a-20805e23eac2-webhook-cert\") pod \"maas-controller-6964775bf7-jdsds\" (UID: \"94cb3a78-9625-40e8-962a-20805e23eac2\") " pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:07:32.140496 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:32.140470 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bb598f85c-ztwgd"] Jul 15 17:07:32.141503 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:07:32.141477 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d6d567e_7555_40be_a78f_cf0b841d5c60.slice/crio-d325e242c68e57e5188584691f38d82b282e1f78f33beec4ddd273a926ec4a3d WatchSource:0}: Error finding container d325e242c68e57e5188584691f38d82b282e1f78f33beec4ddd273a926ec4a3d: Status 404 returned error can't find the container with id d325e242c68e57e5188584691f38d82b282e1f78f33beec4ddd273a926ec4a3d Jul 15 17:07:32.216090 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:32.216062 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:07:32.331950 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:32.331924 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:32.348146 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:32.348124 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6964775bf7-jdsds"] Jul 15 17:07:32.350077 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:07:32.350054 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94cb3a78_9625_40e8_962a_20805e23eac2.slice/crio-d2ea56607f32b021f49b1c414ca6fce6dd511743d1135376a218540ccf734975 WatchSource:0}: Error finding container d2ea56607f32b021f49b1c414ca6fce6dd511743d1135376a218540ccf734975: Status 404 returned error can't find the container with id d2ea56607f32b021f49b1c414ca6fce6dd511743d1135376a218540ccf734975 Jul 15 17:07:32.742900 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:32.742867 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6964775bf7-jdsds" event={"ID":"94cb3a78-9625-40e8-962a-20805e23eac2","Type":"ContainerStarted","Data":"d2ea56607f32b021f49b1c414ca6fce6dd511743d1135376a218540ccf734975"} Jul 15 17:07:32.744032 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:32.744009 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" event={"ID":"8d6d567e-7555-40be-a78f-cf0b841d5c60","Type":"ContainerStarted","Data":"d325e242c68e57e5188584691f38d82b282e1f78f33beec4ddd273a926ec4a3d"} Jul 15 17:07:35.225213 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:35.225168 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:35.327618 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:35.327586 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:35.766820 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:35.766721 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" event={"ID":"8d6d567e-7555-40be-a78f-cf0b841d5c60","Type":"ContainerStarted","Data":"8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91"} Jul 15 17:07:35.767021 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:35.766863 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:35.768458 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:35.768433 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6964775bf7-jdsds" event={"ID":"94cb3a78-9625-40e8-962a-20805e23eac2","Type":"ContainerStarted","Data":"cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886"} Jul 15 17:07:35.768584 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:35.768568 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:07:35.788194 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:35.788153 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" podStartSLOduration=2.888751483 podStartE2EDuration="5.788140054s" totalImagesPullingTime="2.899388571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:07:35.785973077 +0000 UTC m=+506.027968324" watchObservedRunningTime="2026-07-15 17:07:35.788140054 +0000 UTC m=+506.030135526" Jul 15 17:07:35.805699 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:35.805648 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6964775bf7-jdsds" podStartSLOduration=3.10461406 podStartE2EDuration="5.805632536s" totalImagesPullingTime="2.701018476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:07:35.804112372 +0000 UTC m=+506.046107608" watchObservedRunningTime="2026-07-15 17:07:35.805632536 +0000 UTC m=+506.047627762" Jul 15 17:07:46.775776 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:46.775743 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:07:46.776233 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:46.775880 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:46.846897 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:46.846855 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bb598f85c-ztwgd"] Jul 15 17:07:46.847117 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:46.847084 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" podUID="8d6d567e-7555-40be-a78f-cf0b841d5c60" containerName="manager" containerID="cri-o://8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91" gracePeriod=30 Jul 15 17:07:47.105709 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.105686 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:47.127703 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.127673 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pd4j\" (UniqueName: \"kubernetes.io/projected/8d6d567e-7555-40be-a78f-cf0b841d5c60-kube-api-access-8pd4j\") pod \"8d6d567e-7555-40be-a78f-cf0b841d5c60\" (UID: \"8d6d567e-7555-40be-a78f-cf0b841d5c60\") " Jul 15 17:07:47.127830 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.127715 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d6d567e-7555-40be-a78f-cf0b841d5c60-webhook-cert\") pod \"8d6d567e-7555-40be-a78f-cf0b841d5c60\" (UID: \"8d6d567e-7555-40be-a78f-cf0b841d5c60\") " Jul 15 17:07:47.134872 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.130305 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d6d567e-7555-40be-a78f-cf0b841d5c60-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8d6d567e-7555-40be-a78f-cf0b841d5c60" (UID: "8d6d567e-7555-40be-a78f-cf0b841d5c60"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:07:47.134872 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.130309 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d6d567e-7555-40be-a78f-cf0b841d5c60-kube-api-access-8pd4j" (OuterVolumeSpecName: "kube-api-access-8pd4j") pod "8d6d567e-7555-40be-a78f-cf0b841d5c60" (UID: "8d6d567e-7555-40be-a78f-cf0b841d5c60"). InnerVolumeSpecName "kube-api-access-8pd4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:07:47.229192 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.229163 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pd4j\" (UniqueName: \"kubernetes.io/projected/8d6d567e-7555-40be-a78f-cf0b841d5c60-kube-api-access-8pd4j\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:07:47.229192 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.229189 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d6d567e-7555-40be-a78f-cf0b841d5c60-webhook-cert\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:07:47.832893 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.832863 2562 generic.go:356] "Generic (PLEG): container finished" podID="8d6d567e-7555-40be-a78f-cf0b841d5c60" containerID="8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91" exitCode=0 Jul 15 17:07:47.833282 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.832954 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" event={"ID":"8d6d567e-7555-40be-a78f-cf0b841d5c60","Type":"ContainerDied","Data":"8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91"} Jul 15 17:07:47.833282 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.832971 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" Jul 15 17:07:47.833282 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.832992 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb598f85c-ztwgd" event={"ID":"8d6d567e-7555-40be-a78f-cf0b841d5c60","Type":"ContainerDied","Data":"d325e242c68e57e5188584691f38d82b282e1f78f33beec4ddd273a926ec4a3d"} Jul 15 17:07:47.833282 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.833015 2562 scope.go:117] "RemoveContainer" containerID="8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91" Jul 15 17:07:47.842557 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.842536 2562 scope.go:117] "RemoveContainer" containerID="8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91" Jul 15 17:07:47.842797 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:07:47.842776 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91\": container with ID starting with 8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91 not found: ID does not exist" containerID="8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91" Jul 15 17:07:47.842861 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.842806 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91"} err="failed to get container status \"8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91\": rpc error: code = NotFound desc = could not find container \"8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91\": container with ID starting with 8ee51a9d0c4a29a26a7bc5ee7c3ffddca81ba3919200df7d93df4affd0adbb91 not found: ID does not exist" Jul 15 17:07:47.856070 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.856049 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bb598f85c-ztwgd"] Jul 15 17:07:47.860101 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:47.860081 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bb598f85c-ztwgd"] Jul 15 17:07:48.251679 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:48.251646 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d6d567e-7555-40be-a78f-cf0b841d5c60" path="/var/lib/kubelet/pods/8d6d567e-7555-40be-a78f-cf0b841d5c60/volumes" Jul 15 17:07:51.816473 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:51.816421 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:07:51.875367 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:07:51.875330 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:07.670826 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.670751 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc"] Jul 15 17:08:07.671271 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.671124 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d6d567e-7555-40be-a78f-cf0b841d5c60" containerName="manager" Jul 15 17:08:07.671271 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.671136 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d6d567e-7555-40be-a78f-cf0b841d5c60" containerName="manager" Jul 15 17:08:07.671271 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.671195 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d6d567e-7555-40be-a78f-cf0b841d5c60" containerName="manager" Jul 15 17:08:07.676618 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.676601 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:07.680617 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.680590 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:08:07.680862 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.680827 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 15 17:08:07.681808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.681766 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-97jqt\"" Jul 15 17:08:07.681808 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.681769 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 17:08:07.685451 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.685428 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc"] Jul 15 17:08:07.798754 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.798723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69ffda4d-02af-4f53-969a-e98bcec2ae58-maas-api-tls\") pod \"maas-api-595df7b4f4-26wcc\" (UID: \"69ffda4d-02af-4f53-969a-e98bcec2ae58\") " pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:07.798933 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.798761 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9wbd\" (UniqueName: \"kubernetes.io/projected/69ffda4d-02af-4f53-969a-e98bcec2ae58-kube-api-access-w9wbd\") pod \"maas-api-595df7b4f4-26wcc\" (UID: \"69ffda4d-02af-4f53-969a-e98bcec2ae58\") " pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:07.798933 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.798804 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ffda4d-02af-4f53-969a-e98bcec2ae58-openshift-service-ca\") pod \"maas-api-595df7b4f4-26wcc\" (UID: \"69ffda4d-02af-4f53-969a-e98bcec2ae58\") " pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:07.900165 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.900128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ffda4d-02af-4f53-969a-e98bcec2ae58-openshift-service-ca\") pod \"maas-api-595df7b4f4-26wcc\" (UID: \"69ffda4d-02af-4f53-969a-e98bcec2ae58\") " pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:07.900338 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.900218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69ffda4d-02af-4f53-969a-e98bcec2ae58-maas-api-tls\") pod \"maas-api-595df7b4f4-26wcc\" (UID: \"69ffda4d-02af-4f53-969a-e98bcec2ae58\") " pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:07.900338 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.900254 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9wbd\" (UniqueName: \"kubernetes.io/projected/69ffda4d-02af-4f53-969a-e98bcec2ae58-kube-api-access-w9wbd\") pod \"maas-api-595df7b4f4-26wcc\" (UID: \"69ffda4d-02af-4f53-969a-e98bcec2ae58\") " pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:07.900747 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.900724 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ffda4d-02af-4f53-969a-e98bcec2ae58-openshift-service-ca\") pod \"maas-api-595df7b4f4-26wcc\" (UID: \"69ffda4d-02af-4f53-969a-e98bcec2ae58\") " pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:07.902783 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.902763 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69ffda4d-02af-4f53-969a-e98bcec2ae58-maas-api-tls\") pod \"maas-api-595df7b4f4-26wcc\" (UID: \"69ffda4d-02af-4f53-969a-e98bcec2ae58\") " pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:07.909276 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.909251 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9wbd\" (UniqueName: \"kubernetes.io/projected/69ffda4d-02af-4f53-969a-e98bcec2ae58-kube-api-access-w9wbd\") pod \"maas-api-595df7b4f4-26wcc\" (UID: \"69ffda4d-02af-4f53-969a-e98bcec2ae58\") " pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:07.988548 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:07.988471 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:08.135957 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.135930 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc"] Jul 15 17:08:08.138135 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:08:08.138111 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ffda4d_02af_4f53_969a_e98bcec2ae58.slice/crio-f2d3be56f502acf5ec8b4c82ad3710aebc13fa9e22123f5e00379b5b4abb2a6b WatchSource:0}: Error finding container f2d3be56f502acf5ec8b4c82ad3710aebc13fa9e22123f5e00379b5b4abb2a6b: Status 404 returned error can't find the container with id f2d3be56f502acf5ec8b4c82ad3710aebc13fa9e22123f5e00379b5b4abb2a6b Jul 15 17:08:08.269324 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.269250 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt"] Jul 15 17:08:08.273913 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.273894 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" Jul 15 17:08:08.276750 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.276734 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 17:08:08.276891 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.276734 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 17:08:08.277151 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.277131 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 17:08:08.277399 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.277384 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-m986b\"" Jul 15 17:08:08.284918 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.284893 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt"] Jul 15 17:08:08.385897 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.385864 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:08.403935 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.403902 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbff51cd-c23a-4650-ab5a-e9b80283d1de-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-jx5tt\" (UID: \"fbff51cd-c23a-4650-ab5a-e9b80283d1de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" Jul 15 17:08:08.404083 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.404012 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz6wx\" (UniqueName: \"kubernetes.io/projected/fbff51cd-c23a-4650-ab5a-e9b80283d1de-kube-api-access-mz6wx\") pod \"payload-pre-processing-74584bb9f8-jx5tt\" (UID: \"fbff51cd-c23a-4650-ab5a-e9b80283d1de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" Jul 15 17:08:08.422792 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.422764 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-vqxgw"] Jul 15 17:08:08.426108 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.426089 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" Jul 15 17:08:08.437718 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.437694 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-vqxgw"] Jul 15 17:08:08.479417 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.479384 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:08.504630 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.504601 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz6wx\" (UniqueName: \"kubernetes.io/projected/fbff51cd-c23a-4650-ab5a-e9b80283d1de-kube-api-access-mz6wx\") pod \"payload-pre-processing-74584bb9f8-jx5tt\" (UID: \"fbff51cd-c23a-4650-ab5a-e9b80283d1de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" Jul 15 17:08:08.504630 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.504633 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbff51cd-c23a-4650-ab5a-e9b80283d1de-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-jx5tt\" (UID: \"fbff51cd-c23a-4650-ab5a-e9b80283d1de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" Jul 15 17:08:08.504809 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.504672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a81cd00-394e-4fd4-8368-1f2ba0be9e57-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-vqxgw\" (UID: \"0a81cd00-394e-4fd4-8368-1f2ba0be9e57\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" Jul 15 17:08:08.504809 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.504702 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4l2p\" (UniqueName: \"kubernetes.io/projected/0a81cd00-394e-4fd4-8368-1f2ba0be9e57-kube-api-access-c4l2p\") pod \"payload-processing-f9b8d9b9b-vqxgw\" (UID: \"0a81cd00-394e-4fd4-8368-1f2ba0be9e57\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" Jul 15 17:08:08.505646 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.505623 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbff51cd-c23a-4650-ab5a-e9b80283d1de-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-jx5tt\" (UID: \"fbff51cd-c23a-4650-ab5a-e9b80283d1de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" Jul 15 17:08:08.514634 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.514610 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz6wx\" (UniqueName: \"kubernetes.io/projected/fbff51cd-c23a-4650-ab5a-e9b80283d1de-kube-api-access-mz6wx\") pod \"payload-pre-processing-74584bb9f8-jx5tt\" (UID: \"fbff51cd-c23a-4650-ab5a-e9b80283d1de\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" Jul 15 17:08:08.584909 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.584882 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" Jul 15 17:08:08.606204 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.606177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a81cd00-394e-4fd4-8368-1f2ba0be9e57-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-vqxgw\" (UID: \"0a81cd00-394e-4fd4-8368-1f2ba0be9e57\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" Jul 15 17:08:08.606329 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.606231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4l2p\" (UniqueName: \"kubernetes.io/projected/0a81cd00-394e-4fd4-8368-1f2ba0be9e57-kube-api-access-c4l2p\") pod \"payload-processing-f9b8d9b9b-vqxgw\" (UID: \"0a81cd00-394e-4fd4-8368-1f2ba0be9e57\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" Jul 15 17:08:08.606955 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.606935 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a81cd00-394e-4fd4-8368-1f2ba0be9e57-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-vqxgw\" (UID: \"0a81cd00-394e-4fd4-8368-1f2ba0be9e57\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" Jul 15 17:08:08.618828 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.618799 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4l2p\" (UniqueName: \"kubernetes.io/projected/0a81cd00-394e-4fd4-8368-1f2ba0be9e57-kube-api-access-c4l2p\") pod \"payload-processing-f9b8d9b9b-vqxgw\" (UID: \"0a81cd00-394e-4fd4-8368-1f2ba0be9e57\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" Jul 15 17:08:08.723057 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.723033 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt"] Jul 15 17:08:08.724333 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:08:08.724309 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbff51cd_c23a_4650_ab5a_e9b80283d1de.slice/crio-bbe7b85093fa5e143925ff1e66468363db47e7bfcb0a8386051e774867485010 WatchSource:0}: Error finding container bbe7b85093fa5e143925ff1e66468363db47e7bfcb0a8386051e774867485010: Status 404 returned error can't find the container with id bbe7b85093fa5e143925ff1e66468363db47e7bfcb0a8386051e774867485010 Jul 15 17:08:08.738939 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.738918 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" Jul 15 17:08:08.871167 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.871140 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-vqxgw"] Jul 15 17:08:08.873134 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:08:08.873104 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a81cd00_394e_4fd4_8368_1f2ba0be9e57.slice/crio-b1a34b3acc2bd056a5461f752f54afa444dc76bb136818f37f6a92eaad8cb8fc WatchSource:0}: Error finding container b1a34b3acc2bd056a5461f752f54afa444dc76bb136818f37f6a92eaad8cb8fc: Status 404 returned error can't find the container with id b1a34b3acc2bd056a5461f752f54afa444dc76bb136818f37f6a92eaad8cb8fc Jul 15 17:08:08.952906 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.952869 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" event={"ID":"fbff51cd-c23a-4650-ab5a-e9b80283d1de","Type":"ContainerStarted","Data":"bbe7b85093fa5e143925ff1e66468363db47e7bfcb0a8386051e774867485010"} Jul 15 17:08:08.954387 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.954359 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" event={"ID":"69ffda4d-02af-4f53-969a-e98bcec2ae58","Type":"ContainerStarted","Data":"f2d3be56f502acf5ec8b4c82ad3710aebc13fa9e22123f5e00379b5b4abb2a6b"} Jul 15 17:08:08.955696 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:08.955668 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" event={"ID":"0a81cd00-394e-4fd4-8368-1f2ba0be9e57","Type":"ContainerStarted","Data":"b1a34b3acc2bd056a5461f752f54afa444dc76bb136818f37f6a92eaad8cb8fc"} Jul 15 17:08:10.969277 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:10.969237 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" event={"ID":"69ffda4d-02af-4f53-969a-e98bcec2ae58","Type":"ContainerStarted","Data":"ca09a709fffd3c120888daf834cde44a52b6f73eacb2fca097236af4374aaab3"} Jul 15 17:08:10.969726 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:10.969297 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:10.989877 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:10.989763 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" podStartSLOduration=1.401317739 podStartE2EDuration="3.989747463s" totalImagesPullingTime="2.588429724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:10.989025673 +0000 UTC m=+541.231020900" watchObservedRunningTime="2026-07-15 17:08:10.989747463 +0000 UTC m=+541.231742688" Jul 15 17:08:12.984743 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:12.984705 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" event={"ID":"fbff51cd-c23a-4650-ab5a-e9b80283d1de","Type":"ContainerStarted","Data":"c640903973dc9427b731623f51cb83d04a24c2c7d9b1a8e24bfb4d7d72a88a13"} Jul 15 17:08:12.985220 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:12.984791 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" Jul 15 17:08:12.986206 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:12.986181 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" event={"ID":"0a81cd00-394e-4fd4-8368-1f2ba0be9e57","Type":"ContainerStarted","Data":"276edda887a5a7fb48fa9bf8f2f84ffbf93568db53e01319871ebcbcd3618372"} Jul 15 17:08:12.986300 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:12.986243 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" Jul 15 17:08:13.004175 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:13.004131 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" podStartSLOduration=1.560898079 podStartE2EDuration="5.004117926s" totalImagesPullingTime="3.443219847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:13.001920501 +0000 UTC m=+543.243915728" watchObservedRunningTime="2026-07-15 17:08:13.004117926 +0000 UTC m=+543.246113151" Jul 15 17:08:13.022145 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:13.022101 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" podStartSLOduration=1.73021695 podStartE2EDuration="5.022087961s" totalImagesPullingTime="3.291871011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:13.020257311 +0000 UTC m=+543.262252539" watchObservedRunningTime="2026-07-15 17:08:13.022087961 +0000 UTC m=+543.264083186" Jul 15 17:08:16.984403 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:16.984372 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595df7b4f4-26wcc" Jul 15 17:08:18.011446 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:18.011408 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:18.114926 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:18.114893 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:23.992677 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:23.992618 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" podUID="fbff51cd-c23a-4650-ab5a-e9b80283d1de" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.43:9004: connect: connection refused" Jul 15 17:08:23.993078 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:23.992997 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-vqxgw" Jul 15 17:08:27.764670 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:27.764632 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:28.407123 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:28.407088 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:33.731714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.731686 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp"] Jul 15 17:08:33.735221 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.735204 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.738034 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.738009 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4bf9z\"" Jul 15 17:08:33.738133 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.738018 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 17:08:33.738976 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.738960 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 17:08:33.739105 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.739088 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 15 17:08:33.753085 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.753065 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp"] Jul 15 17:08:33.763589 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.763570 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:08:33.828629 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.828599 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.828629 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.828629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.828862 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.828650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.828862 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.828700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.828862 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.828759 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9j9c\" (UniqueName: \"kubernetes.io/projected/25d363eb-fc26-45cc-bca3-8788a082ab4a-kube-api-access-c9j9c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.828862 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.828797 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25d363eb-fc26-45cc-bca3-8788a082ab4a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.929577 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.929531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.929577 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.929572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.929813 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.929602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.929813 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.929729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.929956 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.929813 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9j9c\" (UniqueName: \"kubernetes.io/projected/25d363eb-fc26-45cc-bca3-8788a082ab4a-kube-api-access-c9j9c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.929956 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.929916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25d363eb-fc26-45cc-bca3-8788a082ab4a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.930048 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.930035 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.930098 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.930063 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.930405 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.930380 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.932141 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.932118 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25d363eb-fc26-45cc-bca3-8788a082ab4a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.932453 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.932434 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25d363eb-fc26-45cc-bca3-8788a082ab4a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.938715 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.938691 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9j9c\" (UniqueName: \"kubernetes.io/projected/25d363eb-fc26-45cc-bca3-8788a082ab4a-kube-api-access-c9j9c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wphhp\" (UID: \"25d363eb-fc26-45cc-bca3-8788a082ab4a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:33.992065 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:33.991981 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jx5tt" Jul 15 17:08:34.046104 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:34.046049 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:34.188894 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:34.188863 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp"] Jul 15 17:08:34.189663 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:08:34.189634 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25d363eb_fc26_45cc_bca3_8788a082ab4a.slice/crio-b76f3854d1f6b0cac26302b2b40abc5464d5fe08d921a284b1dd91d9b6f7ab9f WatchSource:0}: Error finding container b76f3854d1f6b0cac26302b2b40abc5464d5fe08d921a284b1dd91d9b6f7ab9f: Status 404 returned error can't find the container with id b76f3854d1f6b0cac26302b2b40abc5464d5fe08d921a284b1dd91d9b6f7ab9f Jul 15 17:08:35.125968 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:35.125930 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" event={"ID":"25d363eb-fc26-45cc-bca3-8788a082ab4a","Type":"ContainerStarted","Data":"b76f3854d1f6b0cac26302b2b40abc5464d5fe08d921a284b1dd91d9b6f7ab9f"} Jul 15 17:08:36.133740 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:36.133703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" event={"ID":"25d363eb-fc26-45cc-bca3-8788a082ab4a","Type":"ContainerStarted","Data":"2fa598d95b18de474628b0876fe231191241033fc9e45e0aa3d1743d9a3fc0d0"} Jul 15 17:08:36.154289 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:36.154244 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" podStartSLOduration=2.067377346 podStartE2EDuration="3.154227709s" totalImagesPullingTime="1.086850363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:08:36.151990902 +0000 UTC m=+566.393986131" watchObservedRunningTime="2026-07-15 17:08:36.154227709 +0000 UTC m=+566.396222934" Jul 15 17:08:44.046930 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:44.046882 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:44.047551 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:44.046995 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:44.059364 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:44.059343 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:08:44.189666 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:08:44.189635 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wphhp" Jul 15 17:09:05.650908 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:05.650866 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:14.725097 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.720700 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6"] Jul 15 17:09:14.727385 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.725797 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.729649 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.729629 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 15 17:09:14.731277 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.731253 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6"] Jul 15 17:09:14.815460 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.815432 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:14.890308 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.890275 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/160677ed-43ed-4ee3-b1fc-0688c32a0f37-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.890583 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.890560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.890756 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.890722 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.890756 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.890756 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.890959 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.890809 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.890959 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.890921 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp7j5\" (UniqueName: \"kubernetes.io/projected/160677ed-43ed-4ee3-b1fc-0688c32a0f37-kube-api-access-hp7j5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.991804 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.991724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp7j5\" (UniqueName: \"kubernetes.io/projected/160677ed-43ed-4ee3-b1fc-0688c32a0f37-kube-api-access-hp7j5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.992089 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.991806 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/160677ed-43ed-4ee3-b1fc-0688c32a0f37-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.992089 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.991870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.992089 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.991901 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.992089 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.991929 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.992089 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.991964 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.992414 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.992393 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.992471 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.992431 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.992471 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.992447 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.994305 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.994287 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/160677ed-43ed-4ee3-b1fc-0688c32a0f37-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:14.994472 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:14.994454 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/160677ed-43ed-4ee3-b1fc-0688c32a0f37-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:15.000704 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:15.000687 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp7j5\" (UniqueName: \"kubernetes.io/projected/160677ed-43ed-4ee3-b1fc-0688c32a0f37-kube-api-access-hp7j5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6\" (UID: \"160677ed-43ed-4ee3-b1fc-0688c32a0f37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:15.037850 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:15.037801 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:15.173683 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:15.173655 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6"] Jul 15 17:09:15.175185 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:09:15.175160 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod160677ed_43ed_4ee3_b1fc_0688c32a0f37.slice/crio-817fbb64906919af713ab2a2055b0a4cfa5289a7cc923fbd51dc634c6613b976 WatchSource:0}: Error finding container 817fbb64906919af713ab2a2055b0a4cfa5289a7cc923fbd51dc634c6613b976: Status 404 returned error can't find the container with id 817fbb64906919af713ab2a2055b0a4cfa5289a7cc923fbd51dc634c6613b976 Jul 15 17:09:15.347423 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:15.347387 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" event={"ID":"160677ed-43ed-4ee3-b1fc-0688c32a0f37","Type":"ContainerStarted","Data":"817fbb64906919af713ab2a2055b0a4cfa5289a7cc923fbd51dc634c6613b976"} Jul 15 17:09:16.354558 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:16.354523 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" event={"ID":"160677ed-43ed-4ee3-b1fc-0688c32a0f37","Type":"ContainerStarted","Data":"1b908e983da93017ca86ff2fe073acf36dc0ec5e8bbb23ed3c63a9064e06ff60"} Jul 15 17:09:16.375132 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:16.375082 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" podStartSLOduration=2.156686208 podStartE2EDuration="2.375067408s" totalImagesPullingTime="218.3812ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:16.372355598 +0000 UTC m=+606.614350829" watchObservedRunningTime="2026-07-15 17:09:16.375067408 +0000 UTC m=+606.617062634" Jul 15 17:09:24.672182 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:24.672138 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:25.038922 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:25.038820 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:25.038922 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:25.038898 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:25.051251 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:25.051228 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:25.422067 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:25.422040 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6" Jul 15 17:09:28.974192 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:28.974160 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:31.927180 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:31.927098 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:49.329399 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.329362 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq"] Jul 15 17:09:49.333108 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.333082 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.335962 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.335942 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 15 17:09:49.344023 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.343999 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq"] Jul 15 17:09:49.350587 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.350564 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:49.498225 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.498186 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.498402 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.498230 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8dd5\" (UniqueName: \"kubernetes.io/projected/f91406d7-1c97-4c45-97dc-f21e6af07f19-kube-api-access-w8dd5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.498402 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.498263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.498493 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.498452 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.498536 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.498493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.498585 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.498554 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f91406d7-1c97-4c45-97dc-f21e6af07f19-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.599742 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.599657 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.599742 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.599698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.599742 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.599725 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f91406d7-1c97-4c45-97dc-f21e6af07f19-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.600058 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.599763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.600058 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.599785 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8dd5\" (UniqueName: \"kubernetes.io/projected/f91406d7-1c97-4c45-97dc-f21e6af07f19-kube-api-access-w8dd5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.600058 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.599806 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.600223 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.600180 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.600283 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.600263 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.600283 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.600274 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.602062 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.602035 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f91406d7-1c97-4c45-97dc-f21e6af07f19-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.602325 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.602298 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f91406d7-1c97-4c45-97dc-f21e6af07f19-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.608448 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.608432 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8dd5\" (UniqueName: \"kubernetes.io/projected/f91406d7-1c97-4c45-97dc-f21e6af07f19-kube-api-access-w8dd5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq\" (UID: \"f91406d7-1c97-4c45-97dc-f21e6af07f19\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.643454 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.643423 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:49.782579 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:49.782554 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq"] Jul 15 17:09:49.783818 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:09:49.783794 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf91406d7_1c97_4c45_97dc_f21e6af07f19.slice/crio-2d9dcc10f46c682e84d8e55f3e86d9037bcc24d84a79dd195dbb226d9a17c164 WatchSource:0}: Error finding container 2d9dcc10f46c682e84d8e55f3e86d9037bcc24d84a79dd195dbb226d9a17c164: Status 404 returned error can't find the container with id 2d9dcc10f46c682e84d8e55f3e86d9037bcc24d84a79dd195dbb226d9a17c164 Jul 15 17:09:50.544489 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:50.544453 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" event={"ID":"f91406d7-1c97-4c45-97dc-f21e6af07f19","Type":"ContainerStarted","Data":"6e13c09899479eaa0fa1b1c42d0d61fd2dc092696fc0199a104c7df86e826fc3"} Jul 15 17:09:50.544489 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:50.544488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" event={"ID":"f91406d7-1c97-4c45-97dc-f21e6af07f19","Type":"ContainerStarted","Data":"2d9dcc10f46c682e84d8e55f3e86d9037bcc24d84a79dd195dbb226d9a17c164"} Jul 15 17:09:54.649047 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:54.649005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:54.749562 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:54.749521 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:09:59.644016 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:59.643982 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:59.644454 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:59.644028 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:59.656686 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:59.656664 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:09:59.677049 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:09:59.677006 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" podStartSLOduration=10.462279551 podStartE2EDuration="10.676991252s" totalImagesPullingTime="214.711701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:50.563262557 +0000 UTC m=+640.805257783" watchObservedRunningTime="2026-07-15 17:09:59.676991252 +0000 UTC m=+649.918986477" Jul 15 17:10:00.609681 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:10:00.609651 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq" Jul 15 17:10:19.566645 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:10:19.566610 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:10:56.715862 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:10:56.715802 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:11:30.696593 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:11:30.696553 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:11:30.823231 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:11:30.823200 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:11:31.848803 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:11:31.848768 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:11:31.946025 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:11:31.945996 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:12:42.398398 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:42.398314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:12:42.532063 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:42.532025 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6964775bf7-jdsds"] Jul 15 17:12:42.532306 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:42.532265 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6964775bf7-jdsds" podUID="94cb3a78-9625-40e8-962a-20805e23eac2" containerName="manager" containerID="cri-o://cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886" gracePeriod=30 Jul 15 17:12:42.788722 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:42.788700 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:12:42.950981 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:42.950885 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94cb3a78-9625-40e8-962a-20805e23eac2-webhook-cert\") pod \"94cb3a78-9625-40e8-962a-20805e23eac2\" (UID: \"94cb3a78-9625-40e8-962a-20805e23eac2\") " Jul 15 17:12:42.950981 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:42.950981 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2q2wk\" (UniqueName: \"kubernetes.io/projected/94cb3a78-9625-40e8-962a-20805e23eac2-kube-api-access-2q2wk\") pod \"94cb3a78-9625-40e8-962a-20805e23eac2\" (UID: \"94cb3a78-9625-40e8-962a-20805e23eac2\") " Jul 15 17:12:42.953124 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:42.953096 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94cb3a78-9625-40e8-962a-20805e23eac2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "94cb3a78-9625-40e8-962a-20805e23eac2" (UID: "94cb3a78-9625-40e8-962a-20805e23eac2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:12:42.953226 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:42.953092 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94cb3a78-9625-40e8-962a-20805e23eac2-kube-api-access-2q2wk" (OuterVolumeSpecName: "kube-api-access-2q2wk") pod "94cb3a78-9625-40e8-962a-20805e23eac2" (UID: "94cb3a78-9625-40e8-962a-20805e23eac2"). InnerVolumeSpecName "kube-api-access-2q2wk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:12:43.051927 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:43.051900 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q2wk\" (UniqueName: \"kubernetes.io/projected/94cb3a78-9625-40e8-962a-20805e23eac2-kube-api-access-2q2wk\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:12:43.051927 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:43.051923 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94cb3a78-9625-40e8-962a-20805e23eac2-webhook-cert\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:12:43.494430 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:43.494395 2562 generic.go:356] "Generic (PLEG): container finished" podID="94cb3a78-9625-40e8-962a-20805e23eac2" containerID="cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886" exitCode=0 Jul 15 17:12:43.494430 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:43.494431 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6964775bf7-jdsds" event={"ID":"94cb3a78-9625-40e8-962a-20805e23eac2","Type":"ContainerDied","Data":"cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886"} Jul 15 17:12:43.494909 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:43.494444 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6964775bf7-jdsds" Jul 15 17:12:43.494909 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:43.494468 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6964775bf7-jdsds" event={"ID":"94cb3a78-9625-40e8-962a-20805e23eac2","Type":"ContainerDied","Data":"d2ea56607f32b021f49b1c414ca6fce6dd511743d1135376a218540ccf734975"} Jul 15 17:12:43.494909 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:43.494488 2562 scope.go:117] "RemoveContainer" containerID="cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886" Jul 15 17:12:43.504335 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:43.504317 2562 scope.go:117] "RemoveContainer" containerID="cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886" Jul 15 17:12:43.504588 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:12:43.504570 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886\": container with ID starting with cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886 not found: ID does not exist" containerID="cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886" Jul 15 17:12:43.504651 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:43.504595 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886"} err="failed to get container status \"cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886\": rpc error: code = NotFound desc = could not find container \"cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886\": container with ID starting with cacd1c31fb8ead367aa1bffe4cf4e1022e217de0c0b072229445bba557849886 not found: ID does not exist" Jul 15 17:12:43.523984 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:43.523959 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6964775bf7-jdsds"] Jul 15 17:12:43.525967 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:43.525947 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6964775bf7-jdsds"] Jul 15 17:12:44.252089 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.252053 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94cb3a78-9625-40e8-962a-20805e23eac2" path="/var/lib/kubelet/pods/94cb3a78-9625-40e8-962a-20805e23eac2/volumes" Jul 15 17:12:44.304209 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.304181 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:12:44.440817 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.440782 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6964775bf7-2npjj"] Jul 15 17:12:44.441407 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.441388 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94cb3a78-9625-40e8-962a-20805e23eac2" containerName="manager" Jul 15 17:12:44.441407 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.441408 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="94cb3a78-9625-40e8-962a-20805e23eac2" containerName="manager" Jul 15 17:12:44.441560 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.441484 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="94cb3a78-9625-40e8-962a-20805e23eac2" containerName="manager" Jul 15 17:12:44.445850 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.445821 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6964775bf7-2npjj" Jul 15 17:12:44.451192 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.451173 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d6x2t\"" Jul 15 17:12:44.451304 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.451185 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 17:12:44.451304 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.451190 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 17:12:44.464889 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.464866 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6964775bf7-2npjj"] Jul 15 17:12:44.565092 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.565003 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcce1a78-bd24-4782-9f24-d14c5839b875-webhook-cert\") pod \"maas-controller-6964775bf7-2npjj\" (UID: \"fcce1a78-bd24-4782-9f24-d14c5839b875\") " pod="opendatahub/maas-controller-6964775bf7-2npjj" Jul 15 17:12:44.565092 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.565039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zf5b\" (UniqueName: \"kubernetes.io/projected/fcce1a78-bd24-4782-9f24-d14c5839b875-kube-api-access-4zf5b\") pod \"maas-controller-6964775bf7-2npjj\" (UID: \"fcce1a78-bd24-4782-9f24-d14c5839b875\") " pod="opendatahub/maas-controller-6964775bf7-2npjj" Jul 15 17:12:44.666376 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.666343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcce1a78-bd24-4782-9f24-d14c5839b875-webhook-cert\") pod \"maas-controller-6964775bf7-2npjj\" (UID: \"fcce1a78-bd24-4782-9f24-d14c5839b875\") " pod="opendatahub/maas-controller-6964775bf7-2npjj" Jul 15 17:12:44.666518 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.666381 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zf5b\" (UniqueName: \"kubernetes.io/projected/fcce1a78-bd24-4782-9f24-d14c5839b875-kube-api-access-4zf5b\") pod \"maas-controller-6964775bf7-2npjj\" (UID: \"fcce1a78-bd24-4782-9f24-d14c5839b875\") " pod="opendatahub/maas-controller-6964775bf7-2npjj" Jul 15 17:12:44.668965 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.668948 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcce1a78-bd24-4782-9f24-d14c5839b875-webhook-cert\") pod \"maas-controller-6964775bf7-2npjj\" (UID: \"fcce1a78-bd24-4782-9f24-d14c5839b875\") " pod="opendatahub/maas-controller-6964775bf7-2npjj" Jul 15 17:12:44.678520 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.678499 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zf5b\" (UniqueName: \"kubernetes.io/projected/fcce1a78-bd24-4782-9f24-d14c5839b875-kube-api-access-4zf5b\") pod \"maas-controller-6964775bf7-2npjj\" (UID: \"fcce1a78-bd24-4782-9f24-d14c5839b875\") " pod="opendatahub/maas-controller-6964775bf7-2npjj" Jul 15 17:12:44.756069 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.756041 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6964775bf7-2npjj" Jul 15 17:12:44.894493 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.894465 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6964775bf7-2npjj"] Jul 15 17:12:44.896175 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:12:44.896152 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcce1a78_bd24_4782_9f24_d14c5839b875.slice/crio-4875d360a654f6cf4a7190be8c7d0521555694b1b54535948f4edd64b861619c WatchSource:0}: Error finding container 4875d360a654f6cf4a7190be8c7d0521555694b1b54535948f4edd64b861619c: Status 404 returned error can't find the container with id 4875d360a654f6cf4a7190be8c7d0521555694b1b54535948f4edd64b861619c Jul 15 17:12:44.897528 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:44.897513 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:12:45.507176 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:45.507139 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6964775bf7-2npjj" event={"ID":"fcce1a78-bd24-4782-9f24-d14c5839b875","Type":"ContainerStarted","Data":"4875d360a654f6cf4a7190be8c7d0521555694b1b54535948f4edd64b861619c"} Jul 15 17:12:46.513653 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:46.513617 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6964775bf7-2npjj" event={"ID":"fcce1a78-bd24-4782-9f24-d14c5839b875","Type":"ContainerStarted","Data":"20ac43ae9b4d507e18a3442e73daadb2ff0301c72b81f5caad3e1abaa3e6763e"} Jul 15 17:12:46.514073 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:46.513689 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6964775bf7-2npjj" Jul 15 17:12:46.533210 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:46.533161 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6964775bf7-2npjj" podStartSLOduration=1.123094732 podStartE2EDuration="2.533140843s" totalImagesPullingTime="1.410046111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:12:46.532759176 +0000 UTC m=+816.774754404" watchObservedRunningTime="2026-07-15 17:12:46.533140843 +0000 UTC m=+816.775136069" Jul 15 17:12:57.521584 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:12:57.521555 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6964775bf7-2npjj" Jul 15 17:13:21.107856 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:13:21.107811 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:21.197033 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:13:21.196999 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:21.356312 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:13:21.356279 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:27.312892 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:13:27.312820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:27.415746 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:13:27.415714 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:27.520101 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:13:27.520049 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:27.782482 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:13:27.782444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:33.851730 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:13:33.851695 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:13:33.937366 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:13:33.937329 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:14:39.691369 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:14:39.691332 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:15:31.874770 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:15:31.874683 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:16:06.587386 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:16:06.587347 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:16:34.159661 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:16:34.159615 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:16:34.258984 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:16:34.258949 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:17:31.709025 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:31.708989 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh"] Jul 15 17:17:31.709530 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:31.709207 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" podUID="11ec1563-96bb-4300-b600-8693ef3738ad" containerName="manager" containerID="cri-o://d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def" gracePeriod=10 Jul 15 17:17:35.270432 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:35.270409 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:17:35.398937 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:35.398904 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11ec1563-96bb-4300-b600-8693ef3738ad-extensions-socket-volume\") pod \"11ec1563-96bb-4300-b600-8693ef3738ad\" (UID: \"11ec1563-96bb-4300-b600-8693ef3738ad\") " Jul 15 17:17:35.399132 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:35.398989 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqs9w\" (UniqueName: \"kubernetes.io/projected/11ec1563-96bb-4300-b600-8693ef3738ad-kube-api-access-sqs9w\") pod \"11ec1563-96bb-4300-b600-8693ef3738ad\" (UID: \"11ec1563-96bb-4300-b600-8693ef3738ad\") " Jul 15 17:17:35.399337 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:35.399312 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11ec1563-96bb-4300-b600-8693ef3738ad-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "11ec1563-96bb-4300-b600-8693ef3738ad" (UID: "11ec1563-96bb-4300-b600-8693ef3738ad"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:17:35.401138 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:35.401114 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11ec1563-96bb-4300-b600-8693ef3738ad-kube-api-access-sqs9w" (OuterVolumeSpecName: "kube-api-access-sqs9w") pod "11ec1563-96bb-4300-b600-8693ef3738ad" (UID: "11ec1563-96bb-4300-b600-8693ef3738ad"). InnerVolumeSpecName "kube-api-access-sqs9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:17:35.499718 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:35.499679 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqs9w\" (UniqueName: \"kubernetes.io/projected/11ec1563-96bb-4300-b600-8693ef3738ad-kube-api-access-sqs9w\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:17:35.499718 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:35.499712 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11ec1563-96bb-4300-b600-8693ef3738ad-extensions-socket-volume\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:17:36.118172 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:36.118138 2562 generic.go:356] "Generic (PLEG): container finished" podID="11ec1563-96bb-4300-b600-8693ef3738ad" containerID="d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def" exitCode=0 Jul 15 17:17:36.118374 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:36.118194 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" Jul 15 17:17:36.118374 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:36.118191 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" event={"ID":"11ec1563-96bb-4300-b600-8693ef3738ad","Type":"ContainerDied","Data":"d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def"} Jul 15 17:17:36.118374 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:36.118317 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh" event={"ID":"11ec1563-96bb-4300-b600-8693ef3738ad","Type":"ContainerDied","Data":"9b00dc06f6e13e6263326b1a18920ac064f68ea0347d6380205393acc15c80e2"} Jul 15 17:17:36.118374 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:36.118338 2562 scope.go:117] "RemoveContainer" containerID="d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def" Jul 15 17:17:36.130004 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:36.129817 2562 scope.go:117] "RemoveContainer" containerID="d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def" Jul 15 17:17:36.130138 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:17:36.130120 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def\": container with ID starting with d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def not found: ID does not exist" containerID="d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def" Jul 15 17:17:36.130182 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:36.130148 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def"} err="failed to get container status \"d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def\": rpc error: code = NotFound desc = could not find container \"d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def\": container with ID starting with d4e6a6010603d9ce7534fbe8a9060a857aada2cd5520483136816e95e3477def not found: ID does not exist" Jul 15 17:17:36.142507 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:36.142484 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh"] Jul 15 17:17:36.147092 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:36.147071 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ppqh"] Jul 15 17:17:36.251744 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:17:36.251718 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11ec1563-96bb-4300-b600-8693ef3738ad" path="/var/lib/kubelet/pods/11ec1563-96bb-4300-b600-8693ef3738ad/volumes" Jul 15 17:18:32.894264 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:32.894193 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn"] Jul 15 17:18:32.894741 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:32.894643 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11ec1563-96bb-4300-b600-8693ef3738ad" containerName="manager" Jul 15 17:18:32.894741 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:32.894661 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="11ec1563-96bb-4300-b600-8693ef3738ad" containerName="manager" Jul 15 17:18:32.894741 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:32.894729 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="11ec1563-96bb-4300-b600-8693ef3738ad" containerName="manager" Jul 15 17:18:32.899002 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:32.898985 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" Jul 15 17:18:32.901771 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:32.901754 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9jw68\"" Jul 15 17:18:32.916784 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:32.916764 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn"] Jul 15 17:18:33.006278 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:33.006250 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8bpv\" (UniqueName: \"kubernetes.io/projected/6aecc0cc-e91a-4acf-86af-a88f46082860-kube-api-access-j8bpv\") pod \"kuadrant-operator-controller-manager-86878c769f-zgnrn\" (UID: \"6aecc0cc-e91a-4acf-86af-a88f46082860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" Jul 15 17:18:33.006402 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:33.006309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6aecc0cc-e91a-4acf-86af-a88f46082860-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zgnrn\" (UID: \"6aecc0cc-e91a-4acf-86af-a88f46082860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" Jul 15 17:18:33.107208 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:33.107179 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8bpv\" (UniqueName: \"kubernetes.io/projected/6aecc0cc-e91a-4acf-86af-a88f46082860-kube-api-access-j8bpv\") pod \"kuadrant-operator-controller-manager-86878c769f-zgnrn\" (UID: \"6aecc0cc-e91a-4acf-86af-a88f46082860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" Jul 15 17:18:33.107366 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:33.107239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6aecc0cc-e91a-4acf-86af-a88f46082860-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zgnrn\" (UID: \"6aecc0cc-e91a-4acf-86af-a88f46082860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" Jul 15 17:18:33.107598 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:33.107581 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6aecc0cc-e91a-4acf-86af-a88f46082860-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zgnrn\" (UID: \"6aecc0cc-e91a-4acf-86af-a88f46082860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" Jul 15 17:18:33.121116 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:33.121092 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8bpv\" (UniqueName: \"kubernetes.io/projected/6aecc0cc-e91a-4acf-86af-a88f46082860-kube-api-access-j8bpv\") pod \"kuadrant-operator-controller-manager-86878c769f-zgnrn\" (UID: \"6aecc0cc-e91a-4acf-86af-a88f46082860\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" Jul 15 17:18:33.208973 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:33.208916 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" Jul 15 17:18:33.372148 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:33.372126 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn"] Jul 15 17:18:33.374883 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:18:33.374827 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aecc0cc_e91a_4acf_86af_a88f46082860.slice/crio-e5491dfa51d88cac3f88e04b9e467a648ad52bcc2c3627408da88ed014d78717 WatchSource:0}: Error finding container e5491dfa51d88cac3f88e04b9e467a648ad52bcc2c3627408da88ed014d78717: Status 404 returned error can't find the container with id e5491dfa51d88cac3f88e04b9e467a648ad52bcc2c3627408da88ed014d78717 Jul 15 17:18:33.427091 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:33.427041 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" event={"ID":"6aecc0cc-e91a-4acf-86af-a88f46082860","Type":"ContainerStarted","Data":"e5491dfa51d88cac3f88e04b9e467a648ad52bcc2c3627408da88ed014d78717"} Jul 15 17:18:34.434701 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:34.434651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" event={"ID":"6aecc0cc-e91a-4acf-86af-a88f46082860","Type":"ContainerStarted","Data":"676407e9e2cc9feb92b394bad2c3f1fbac655ab1fa59a4ffb40e51ce89c86d13"} Jul 15 17:18:34.435159 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:34.434875 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" Jul 15 17:18:34.468705 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:34.468654 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" podStartSLOduration=2.468640634 podStartE2EDuration="2.468640634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:18:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:18:34.466997151 +0000 UTC m=+1164.708992377" watchObservedRunningTime="2026-07-15 17:18:34.468640634 +0000 UTC m=+1164.710635859" Jul 15 17:18:45.443547 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:18:45.443513 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zgnrn" Jul 15 17:21:53.476763 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:21:53.476678 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:23:10.769896 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:23:10.769785 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:23:10.870136 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:23:10.870104 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:23:56.973127 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:23:56.973082 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:20.034428 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:20.034391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:27.424727 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:27.424646 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:35.704575 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:35.704491 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:40.837049 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:40.837008 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:43.913204 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:43.913166 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:44.013926 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:44.013895 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:44.394899 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:44.394846 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:50.268225 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:50.268181 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:50.341782 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:50.341747 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:50.857640 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:50.857605 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:50.920297 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:50.920256 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:51.667537 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.667499 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:51.746482 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.746449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:51.752970 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.752941 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6"] Jul 15 17:24:51.756469 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.756449 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:51.758936 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.758919 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-554abb3e-serving-cert\"" Jul 15 17:24:51.768304 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.768283 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6"] Jul 15 17:24:51.809279 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.809252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-maas-api-tls\") pod \"maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:51.809407 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.809302 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrblt\" (UniqueName: \"kubernetes.io/projected/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-kube-api-access-nrblt\") pod \"maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:51.809407 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.809356 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-openshift-service-ca\") pod \"maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:51.816737 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.816709 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:51.910212 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.910174 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-maas-api-tls\") pod \"maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:51.910212 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.910213 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrblt\" (UniqueName: \"kubernetes.io/projected/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-kube-api-access-nrblt\") pod \"maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:51.910439 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.910254 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-openshift-service-ca\") pod \"maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:51.910439 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:24:51.910322 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-serving-cert: secret "maas-api-e2e-ait-554abb3e-serving-cert" not found Jul 15 17:24:51.910439 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:24:51.910398 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-maas-api-tls podName:d65b3cb7-7275-47df-b10a-b6f49ffe07fa nodeName:}" failed. No retries permitted until 2026-07-15 17:24:52.410377054 +0000 UTC m=+1542.652372259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-maas-api-tls") pod "maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" (UID: "d65b3cb7-7275-47df-b10a-b6f49ffe07fa") : secret "maas-api-e2e-ait-554abb3e-serving-cert" not found Jul 15 17:24:51.910968 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.910952 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-openshift-service-ca\") pod \"maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:51.918777 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:51.918721 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrblt\" (UniqueName: \"kubernetes.io/projected/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-kube-api-access-nrblt\") pod \"maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:52.041616 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:52.041583 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:52.351175 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:52.351139 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:52.413906 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:52.413872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-maas-api-tls\") pod \"maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:52.416392 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:52.416369 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-maas-api-tls\") pod \"maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:52.518518 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:52.518485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:52.669025 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:52.668943 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:52.815823 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:52.815797 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6"] Jul 15 17:24:52.820138 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:24:52.820103 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd65b3cb7_7275_47df_b10a_b6f49ffe07fa.slice/crio-06b966e35ffc478c9894cf2edcc11da6d0897636e5a9d533a583d5f39cf6cccd WatchSource:0}: Error finding container 06b966e35ffc478c9894cf2edcc11da6d0897636e5a9d533a583d5f39cf6cccd: Status 404 returned error can't find the container with id 06b966e35ffc478c9894cf2edcc11da6d0897636e5a9d533a583d5f39cf6cccd Jul 15 17:24:52.821139 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:52.821115 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:24:53.510904 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:53.510864 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" event={"ID":"d65b3cb7-7275-47df-b10a-b6f49ffe07fa","Type":"ContainerStarted","Data":"5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12"} Jul 15 17:24:53.511072 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:53.510920 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" event={"ID":"d65b3cb7-7275-47df-b10a-b6f49ffe07fa","Type":"ContainerStarted","Data":"06b966e35ffc478c9894cf2edcc11da6d0897636e5a9d533a583d5f39cf6cccd"} Jul 15 17:24:53.511072 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:53.511060 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:24:53.533345 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:53.533289 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" podStartSLOduration=1.995625472 podStartE2EDuration="2.533269877s" totalImagesPullingTime="537.644405ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:24:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:24:53.528987019 +0000 UTC m=+1543.770982252" watchObservedRunningTime="2026-07-15 17:24:53.533269877 +0000 UTC m=+1543.775265104" Jul 15 17:24:53.904502 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:53.904459 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:24:59.521903 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:24:59.521872 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:25:57.231670 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.231634 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6"] Jul 15 17:25:57.234345 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.231910 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" podUID="d65b3cb7-7275-47df-b10a-b6f49ffe07fa" containerName="maas-api" containerID="cri-o://5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12" gracePeriod=30 Jul 15 17:25:57.253163 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.253115 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:25:57.486490 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.486423 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:25:57.603656 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.603619 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-openshift-service-ca\") pod \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " Jul 15 17:25:57.603891 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.603682 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-maas-api-tls\") pod \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " Jul 15 17:25:57.603891 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.603699 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrblt\" (UniqueName: \"kubernetes.io/projected/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-kube-api-access-nrblt\") pod \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\" (UID: \"d65b3cb7-7275-47df-b10a-b6f49ffe07fa\") " Jul 15 17:25:57.604101 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.604076 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d65b3cb7-7275-47df-b10a-b6f49ffe07fa" (UID: "d65b3cb7-7275-47df-b10a-b6f49ffe07fa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:25:57.606148 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.606118 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-kube-api-access-nrblt" (OuterVolumeSpecName: "kube-api-access-nrblt") pod "d65b3cb7-7275-47df-b10a-b6f49ffe07fa" (UID: "d65b3cb7-7275-47df-b10a-b6f49ffe07fa"). InnerVolumeSpecName "kube-api-access-nrblt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:25:57.606258 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.606147 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d65b3cb7-7275-47df-b10a-b6f49ffe07fa" (UID: "d65b3cb7-7275-47df-b10a-b6f49ffe07fa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:25:57.705225 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.705190 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-openshift-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:25:57.705225 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.705217 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-maas-api-tls\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:25:57.705225 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.705228 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrblt\" (UniqueName: \"kubernetes.io/projected/d65b3cb7-7275-47df-b10a-b6f49ffe07fa-kube-api-access-nrblt\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:25:57.859609 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.859571 2562 generic.go:356] "Generic (PLEG): container finished" podID="d65b3cb7-7275-47df-b10a-b6f49ffe07fa" containerID="5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12" exitCode=0 Jul 15 17:25:57.859788 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.859618 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" Jul 15 17:25:57.859788 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.859649 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" event={"ID":"d65b3cb7-7275-47df-b10a-b6f49ffe07fa","Type":"ContainerDied","Data":"5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12"} Jul 15 17:25:57.859788 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.859700 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6" event={"ID":"d65b3cb7-7275-47df-b10a-b6f49ffe07fa","Type":"ContainerDied","Data":"06b966e35ffc478c9894cf2edcc11da6d0897636e5a9d533a583d5f39cf6cccd"} Jul 15 17:25:57.859788 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.859723 2562 scope.go:117] "RemoveContainer" containerID="5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12" Jul 15 17:25:57.869152 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.869133 2562 scope.go:117] "RemoveContainer" containerID="5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12" Jul 15 17:25:57.869402 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:25:57.869384 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12\": container with ID starting with 5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12 not found: ID does not exist" containerID="5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12" Jul 15 17:25:57.869449 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.869410 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12"} err="failed to get container status \"5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12\": rpc error: code = NotFound desc = could not find container \"5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12\": container with ID starting with 5718ebac491476a8ac982cbb50a483066488238c647554d9d1ab473070dbad12 not found: ID does not exist" Jul 15 17:25:57.882458 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.882429 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6"] Jul 15 17:25:57.888109 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:57.888090 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-554abb3e-6896ddd59b-stfx6"] Jul 15 17:25:58.252625 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:25:58.252552 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d65b3cb7-7275-47df-b10a-b6f49ffe07fa" path="/var/lib/kubelet/pods/d65b3cb7-7275-47df-b10a-b6f49ffe07fa/volumes" Jul 15 17:26:01.860525 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:01.860420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:08.090092 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:08.090062 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:08.154803 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:08.154770 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:12.063999 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:12.063956 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:12.435827 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:12.435787 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:12.525809 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:12.525781 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:12.628303 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:12.628271 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:13.865432 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:13.865388 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:13.953823 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:13.953785 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7"] Jul 15 17:26:13.954457 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:13.954437 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d65b3cb7-7275-47df-b10a-b6f49ffe07fa" containerName="maas-api" Jul 15 17:26:13.954504 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:13.954462 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d65b3cb7-7275-47df-b10a-b6f49ffe07fa" containerName="maas-api" Jul 15 17:26:13.954565 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:13.954554 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d65b3cb7-7275-47df-b10a-b6f49ffe07fa" containerName="maas-api" Jul 15 17:26:13.958082 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:13.958064 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:13.960805 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:13.960786 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-a2500ffb-serving-cert\"" Jul 15 17:26:13.970257 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:13.970236 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7"] Jul 15 17:26:13.995026 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:13.994997 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:14.052392 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.052358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45fed81b-0fc5-459c-bc8d-fde36795221d-openshift-service-ca\") pod \"maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:14.052585 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.052489 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2n9q\" (UniqueName: \"kubernetes.io/projected/45fed81b-0fc5-459c-bc8d-fde36795221d-kube-api-access-l2n9q\") pod \"maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:14.052585 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.052535 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45fed81b-0fc5-459c-bc8d-fde36795221d-maas-api-tls\") pod \"maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:14.153715 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.153614 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2n9q\" (UniqueName: \"kubernetes.io/projected/45fed81b-0fc5-459c-bc8d-fde36795221d-kube-api-access-l2n9q\") pod \"maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:14.153715 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.153667 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45fed81b-0fc5-459c-bc8d-fde36795221d-maas-api-tls\") pod \"maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:14.153715 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.153718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45fed81b-0fc5-459c-bc8d-fde36795221d-openshift-service-ca\") pod \"maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:14.154001 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:26:14.153827 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-serving-cert: secret "maas-api-e2e-migrate-a2500ffb-serving-cert" not found Jul 15 17:26:14.154001 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:26:14.153918 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45fed81b-0fc5-459c-bc8d-fde36795221d-maas-api-tls podName:45fed81b-0fc5-459c-bc8d-fde36795221d nodeName:}" failed. No retries permitted until 2026-07-15 17:26:14.65390127 +0000 UTC m=+1624.895896475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/45fed81b-0fc5-459c-bc8d-fde36795221d-maas-api-tls") pod "maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" (UID: "45fed81b-0fc5-459c-bc8d-fde36795221d") : secret "maas-api-e2e-migrate-a2500ffb-serving-cert" not found Jul 15 17:26:14.154337 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.154319 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45fed81b-0fc5-459c-bc8d-fde36795221d-openshift-service-ca\") pod \"maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:14.162802 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.162779 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2n9q\" (UniqueName: \"kubernetes.io/projected/45fed81b-0fc5-459c-bc8d-fde36795221d-kube-api-access-l2n9q\") pod \"maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:14.216709 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.216677 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:14.530419 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.530333 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:14.632155 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.632118 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:14.659291 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.659258 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45fed81b-0fc5-459c-bc8d-fde36795221d-maas-api-tls\") pod \"maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:14.661790 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.661767 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45fed81b-0fc5-459c-bc8d-fde36795221d-maas-api-tls\") pod \"maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:14.870363 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:14.870328 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:15.010871 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:15.010771 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7"] Jul 15 17:26:15.013593 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:26:15.013562 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45fed81b_0fc5_459c_bc8d_fde36795221d.slice/crio-6554696b2dfa1dbad35a18629e9652918f2aaeea7285068865182a16d99d9bc6 WatchSource:0}: Error finding container 6554696b2dfa1dbad35a18629e9652918f2aaeea7285068865182a16d99d9bc6: Status 404 returned error can't find the container with id 6554696b2dfa1dbad35a18629e9652918f2aaeea7285068865182a16d99d9bc6 Jul 15 17:26:15.961148 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:15.961114 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" event={"ID":"45fed81b-0fc5-459c-bc8d-fde36795221d","Type":"ContainerStarted","Data":"8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb"} Jul 15 17:26:15.961541 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:15.961192 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" event={"ID":"45fed81b-0fc5-459c-bc8d-fde36795221d","Type":"ContainerStarted","Data":"6554696b2dfa1dbad35a18629e9652918f2aaeea7285068865182a16d99d9bc6"} Jul 15 17:26:15.961541 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:15.961262 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:15.980959 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:15.980911 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" podStartSLOduration=2.5048893899999998 podStartE2EDuration="2.980898834s" totalImagesPullingTime="476.009444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:26:15.978702432 +0000 UTC m=+1626.220697658" watchObservedRunningTime="2026-07-15 17:26:15.980898834 +0000 UTC m=+1626.222894059" Jul 15 17:26:18.719760 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:18.719728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:18.780755 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:18.780724 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:26:21.972698 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:21.972665 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:26:22.269229 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:26:22.269199 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:18.501778 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:18.501737 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:18.889565 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:18.889208 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7"] Jul 15 17:27:18.889565 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:18.889472 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" podUID="45fed81b-0fc5-459c-bc8d-fde36795221d" containerName="maas-api" containerID="cri-o://8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb" gracePeriod=30 Jul 15 17:27:19.141218 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.141154 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:27:19.241404 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.241376 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45fed81b-0fc5-459c-bc8d-fde36795221d-openshift-service-ca\") pod \"45fed81b-0fc5-459c-bc8d-fde36795221d\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " Jul 15 17:27:19.241583 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.241443 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2n9q\" (UniqueName: \"kubernetes.io/projected/45fed81b-0fc5-459c-bc8d-fde36795221d-kube-api-access-l2n9q\") pod \"45fed81b-0fc5-459c-bc8d-fde36795221d\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " Jul 15 17:27:19.241583 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.241488 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45fed81b-0fc5-459c-bc8d-fde36795221d-maas-api-tls\") pod \"45fed81b-0fc5-459c-bc8d-fde36795221d\" (UID: \"45fed81b-0fc5-459c-bc8d-fde36795221d\") " Jul 15 17:27:19.241728 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.241704 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45fed81b-0fc5-459c-bc8d-fde36795221d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "45fed81b-0fc5-459c-bc8d-fde36795221d" (UID: "45fed81b-0fc5-459c-bc8d-fde36795221d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:27:19.243677 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.243646 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45fed81b-0fc5-459c-bc8d-fde36795221d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "45fed81b-0fc5-459c-bc8d-fde36795221d" (UID: "45fed81b-0fc5-459c-bc8d-fde36795221d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:27:19.243783 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.243719 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45fed81b-0fc5-459c-bc8d-fde36795221d-kube-api-access-l2n9q" (OuterVolumeSpecName: "kube-api-access-l2n9q") pod "45fed81b-0fc5-459c-bc8d-fde36795221d" (UID: "45fed81b-0fc5-459c-bc8d-fde36795221d"). InnerVolumeSpecName "kube-api-access-l2n9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:27:19.309199 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.309168 2562 generic.go:356] "Generic (PLEG): container finished" podID="45fed81b-0fc5-459c-bc8d-fde36795221d" containerID="8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb" exitCode=0 Jul 15 17:27:19.309339 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.309211 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" Jul 15 17:27:19.309339 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.309250 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" event={"ID":"45fed81b-0fc5-459c-bc8d-fde36795221d","Type":"ContainerDied","Data":"8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb"} Jul 15 17:27:19.309339 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.309298 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7" event={"ID":"45fed81b-0fc5-459c-bc8d-fde36795221d","Type":"ContainerDied","Data":"6554696b2dfa1dbad35a18629e9652918f2aaeea7285068865182a16d99d9bc6"} Jul 15 17:27:19.309339 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.309319 2562 scope.go:117] "RemoveContainer" containerID="8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb" Jul 15 17:27:19.318496 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.318479 2562 scope.go:117] "RemoveContainer" containerID="8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb" Jul 15 17:27:19.318749 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:27:19.318732 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb\": container with ID starting with 8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb not found: ID does not exist" containerID="8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb" Jul 15 17:27:19.318796 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.318758 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb"} err="failed to get container status \"8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb\": rpc error: code = NotFound desc = could not find container \"8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb\": container with ID starting with 8600f11c61c999c08e8edf58b651130a57e0b73bba6b84846b526648030a74cb not found: ID does not exist" Jul 15 17:27:19.331649 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.331627 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7"] Jul 15 17:27:19.336007 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.335988 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a2500ffb-667686fbb8-jl4x7"] Jul 15 17:27:19.343064 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.343032 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2n9q\" (UniqueName: \"kubernetes.io/projected/45fed81b-0fc5-459c-bc8d-fde36795221d-kube-api-access-l2n9q\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:27:19.343064 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.343059 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45fed81b-0fc5-459c-bc8d-fde36795221d-maas-api-tls\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:27:19.343203 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.343074 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45fed81b-0fc5-459c-bc8d-fde36795221d-openshift-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:27:19.998544 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:19.998508 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:20.252289 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:20.252211 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45fed81b-0fc5-459c-bc8d-fde36795221d" path="/var/lib/kubelet/pods/45fed81b-0fc5-459c-bc8d-fde36795221d/volumes" Jul 15 17:27:24.764643 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:24.764604 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:24.835917 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:24.835873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:25.000383 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:25.000342 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:25.256091 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:25.256032 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:25.355450 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:25.355411 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:26.099015 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.098964 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:26.179317 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.179284 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw"] Jul 15 17:27:26.179959 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.179938 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45fed81b-0fc5-459c-bc8d-fde36795221d" containerName="maas-api" Jul 15 17:27:26.179959 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.179958 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="45fed81b-0fc5-459c-bc8d-fde36795221d" containerName="maas-api" Jul 15 17:27:26.180139 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.180081 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="45fed81b-0fc5-459c-bc8d-fde36795221d" containerName="maas-api" Jul 15 17:27:26.184693 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.184668 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:26.187434 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.187402 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-297e6a4a-serving-cert\"" Jul 15 17:27:26.191085 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.191062 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw"] Jul 15 17:27:26.197652 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.197618 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:26.265022 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.264982 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:26.306382 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.306357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb4zc\" (UniqueName: \"kubernetes.io/projected/07b70766-087d-4aa6-b492-cbcbf91ae391-kube-api-access-fb4zc\") pod \"maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:26.306569 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.306547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b70766-087d-4aa6-b492-cbcbf91ae391-openshift-service-ca\") pod \"maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:26.306623 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.306586 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b70766-087d-4aa6-b492-cbcbf91ae391-maas-api-tls\") pod \"maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:26.364043 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.363958 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:26.407054 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.407028 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb4zc\" (UniqueName: \"kubernetes.io/projected/07b70766-087d-4aa6-b492-cbcbf91ae391-kube-api-access-fb4zc\") pod \"maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:26.407190 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.407152 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b70766-087d-4aa6-b492-cbcbf91ae391-openshift-service-ca\") pod \"maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:26.407190 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.407182 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b70766-087d-4aa6-b492-cbcbf91ae391-maas-api-tls\") pod \"maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:26.407742 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.407722 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b70766-087d-4aa6-b492-cbcbf91ae391-openshift-service-ca\") pod \"maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:26.409774 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.409737 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b70766-087d-4aa6-b492-cbcbf91ae391-maas-api-tls\") pod \"maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:26.420274 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.420255 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb4zc\" (UniqueName: \"kubernetes.io/projected/07b70766-087d-4aa6-b492-cbcbf91ae391-kube-api-access-fb4zc\") pod \"maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:26.497544 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.497527 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:26.647684 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.647663 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw"] Jul 15 17:27:26.649630 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:27:26.649599 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b70766_087d_4aa6_b492_cbcbf91ae391.slice/crio-d1bcf8733eed115eb1b58c9a15be9b546b4be1a798f8ca3539d225cf6d19de7f WatchSource:0}: Error finding container d1bcf8733eed115eb1b58c9a15be9b546b4be1a798f8ca3539d225cf6d19de7f: Status 404 returned error can't find the container with id d1bcf8733eed115eb1b58c9a15be9b546b4be1a798f8ca3539d225cf6d19de7f Jul 15 17:27:26.928955 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:26.928855 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:27.015517 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:27.015480 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:27.357510 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:27.357474 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" event={"ID":"07b70766-087d-4aa6-b492-cbcbf91ae391","Type":"ContainerStarted","Data":"815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89"} Jul 15 17:27:27.357912 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:27.357586 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:27.357912 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:27.357605 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" event={"ID":"07b70766-087d-4aa6-b492-cbcbf91ae391","Type":"ContainerStarted","Data":"d1bcf8733eed115eb1b58c9a15be9b546b4be1a798f8ca3539d225cf6d19de7f"} Jul 15 17:27:27.377219 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:27.377176 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" podStartSLOduration=0.86644749 podStartE2EDuration="1.377159372s" totalImagesPullingTime="510.711882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:27:27.374684407 +0000 UTC m=+1697.616679638" watchObservedRunningTime="2026-07-15 17:27:27.377159372 +0000 UTC m=+1697.619154598" Jul 15 17:27:28.644888 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:28.644850 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:33.369003 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:33.368919 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:27:55.294874 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:55.294817 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:27:55.399710 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:27:55.399677 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:09.086943 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:09.086911 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:09.333714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:09.333684 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:31.940060 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:31.940017 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:32.041039 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:32.041003 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:41.726859 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:41.726495 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw"] Jul 15 17:28:41.727342 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:41.727036 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" podUID="07b70766-087d-4aa6-b492-cbcbf91ae391" containerName="maas-api" containerID="cri-o://815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89" gracePeriod=30 Jul 15 17:28:41.754711 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:41.754657 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:41.974613 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:41.974590 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:28:42.061448 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.061373 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb4zc\" (UniqueName: \"kubernetes.io/projected/07b70766-087d-4aa6-b492-cbcbf91ae391-kube-api-access-fb4zc\") pod \"07b70766-087d-4aa6-b492-cbcbf91ae391\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " Jul 15 17:28:42.061631 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.061471 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b70766-087d-4aa6-b492-cbcbf91ae391-maas-api-tls\") pod \"07b70766-087d-4aa6-b492-cbcbf91ae391\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " Jul 15 17:28:42.061631 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.061536 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b70766-087d-4aa6-b492-cbcbf91ae391-openshift-service-ca\") pod \"07b70766-087d-4aa6-b492-cbcbf91ae391\" (UID: \"07b70766-087d-4aa6-b492-cbcbf91ae391\") " Jul 15 17:28:42.061927 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.061901 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07b70766-087d-4aa6-b492-cbcbf91ae391-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "07b70766-087d-4aa6-b492-cbcbf91ae391" (UID: "07b70766-087d-4aa6-b492-cbcbf91ae391"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:28:42.063522 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.063495 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b70766-087d-4aa6-b492-cbcbf91ae391-kube-api-access-fb4zc" (OuterVolumeSpecName: "kube-api-access-fb4zc") pod "07b70766-087d-4aa6-b492-cbcbf91ae391" (UID: "07b70766-087d-4aa6-b492-cbcbf91ae391"). InnerVolumeSpecName "kube-api-access-fb4zc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:28:42.063617 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.063547 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07b70766-087d-4aa6-b492-cbcbf91ae391-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "07b70766-087d-4aa6-b492-cbcbf91ae391" (UID: "07b70766-087d-4aa6-b492-cbcbf91ae391"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:28:42.162897 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.162868 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb4zc\" (UniqueName: \"kubernetes.io/projected/07b70766-087d-4aa6-b492-cbcbf91ae391-kube-api-access-fb4zc\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:28:42.162897 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.162893 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b70766-087d-4aa6-b492-cbcbf91ae391-maas-api-tls\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:28:42.162897 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.162903 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b70766-087d-4aa6-b492-cbcbf91ae391-openshift-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:28:42.783615 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.783579 2562 generic.go:356] "Generic (PLEG): container finished" podID="07b70766-087d-4aa6-b492-cbcbf91ae391" containerID="815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89" exitCode=0 Jul 15 17:28:42.783615 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.783620 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" Jul 15 17:28:42.784154 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.783669 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" event={"ID":"07b70766-087d-4aa6-b492-cbcbf91ae391","Type":"ContainerDied","Data":"815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89"} Jul 15 17:28:42.784154 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.783728 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw" event={"ID":"07b70766-087d-4aa6-b492-cbcbf91ae391","Type":"ContainerDied","Data":"d1bcf8733eed115eb1b58c9a15be9b546b4be1a798f8ca3539d225cf6d19de7f"} Jul 15 17:28:42.784154 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.783763 2562 scope.go:117] "RemoveContainer" containerID="815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89" Jul 15 17:28:42.792882 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.792863 2562 scope.go:117] "RemoveContainer" containerID="815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89" Jul 15 17:28:42.793145 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:28:42.793127 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89\": container with ID starting with 815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89 not found: ID does not exist" containerID="815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89" Jul 15 17:28:42.793199 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.793153 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89"} err="failed to get container status \"815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89\": rpc error: code = NotFound desc = could not find container \"815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89\": container with ID starting with 815f02739aa05a92816b1bee28b998ad5f3c441e0600ef750cdf88fafbaf0e89 not found: ID does not exist" Jul 15 17:28:42.802026 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.802004 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw"] Jul 15 17:28:42.805998 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:42.805976 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-297e6a4a-596b8bc7db-jtzpw"] Jul 15 17:28:44.252490 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:44.252455 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07b70766-087d-4aa6-b492-cbcbf91ae391" path="/var/lib/kubelet/pods/07b70766-087d-4aa6-b492-cbcbf91ae391/volumes" Jul 15 17:28:46.357894 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:46.357861 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:52.526958 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:52.526930 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:52.630137 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:52.630106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:57.400275 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:57.400234 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:57.712341 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:57.712293 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:58.913460 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:58.913415 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:59.004777 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.004749 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn"] Jul 15 17:28:59.005179 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.005166 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b70766-087d-4aa6-b492-cbcbf91ae391" containerName="maas-api" Jul 15 17:28:59.005227 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.005181 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b70766-087d-4aa6-b492-cbcbf91ae391" containerName="maas-api" Jul 15 17:28:59.005263 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.005235 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b70766-087d-4aa6-b492-cbcbf91ae391" containerName="maas-api" Jul 15 17:28:59.008532 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.008514 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:28:59.011303 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.011279 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-6feb2776-serving-cert\"" Jul 15 17:28:59.013871 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.013824 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:59.020599 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.020574 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn"] Jul 15 17:28:59.089645 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.089617 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:59.108811 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.108786 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db391423-e64a-4fbd-bed2-3b09f57ded38-maas-api-tls\") pod \"maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:28:59.108955 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.108812 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5ksf\" (UniqueName: \"kubernetes.io/projected/db391423-e64a-4fbd-bed2-3b09f57ded38-kube-api-access-d5ksf\") pod \"maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:28:59.108955 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.108902 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db391423-e64a-4fbd-bed2-3b09f57ded38-openshift-service-ca\") pod \"maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:28:59.209941 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.209872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db391423-e64a-4fbd-bed2-3b09f57ded38-maas-api-tls\") pod \"maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:28:59.209941 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.209908 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5ksf\" (UniqueName: \"kubernetes.io/projected/db391423-e64a-4fbd-bed2-3b09f57ded38-kube-api-access-d5ksf\") pod \"maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:28:59.210095 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.209940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db391423-e64a-4fbd-bed2-3b09f57ded38-openshift-service-ca\") pod \"maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:28:59.210095 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:28:59.209990 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-serving-cert: secret "maas-api-e2e-derive-6feb2776-serving-cert" not found Jul 15 17:28:59.210095 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:28:59.210046 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db391423-e64a-4fbd-bed2-3b09f57ded38-maas-api-tls podName:db391423-e64a-4fbd-bed2-3b09f57ded38 nodeName:}" failed. No retries permitted until 2026-07-15 17:28:59.710030769 +0000 UTC m=+1789.952025975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db391423-e64a-4fbd-bed2-3b09f57ded38-maas-api-tls") pod "maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" (UID: "db391423-e64a-4fbd-bed2-3b09f57ded38") : secret "maas-api-e2e-derive-6feb2776-serving-cert" not found Jul 15 17:28:59.210542 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.210524 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db391423-e64a-4fbd-bed2-3b09f57ded38-openshift-service-ca\") pod \"maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:28:59.220682 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.220660 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5ksf\" (UniqueName: \"kubernetes.io/projected/db391423-e64a-4fbd-bed2-3b09f57ded38-kube-api-access-d5ksf\") pod \"maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:28:59.344299 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.344264 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:59.601758 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.601721 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:59.695650 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.695614 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:28:59.715755 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.715718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db391423-e64a-4fbd-bed2-3b09f57ded38-maas-api-tls\") pod \"maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:28:59.718420 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.718395 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db391423-e64a-4fbd-bed2-3b09f57ded38-maas-api-tls\") pod \"maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:28:59.921423 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:28:59.921348 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:29:00.069678 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:29:00.069650 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn"] Jul 15 17:29:00.071305 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:29:00.071278 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb391423_e64a_4fbd_bed2_3b09f57ded38.slice/crio-eb4df14b1a1ed1b0f51623299752e499fb35e348cebe98376a23fc78002c10f6 WatchSource:0}: Error finding container eb4df14b1a1ed1b0f51623299752e499fb35e348cebe98376a23fc78002c10f6: Status 404 returned error can't find the container with id eb4df14b1a1ed1b0f51623299752e499fb35e348cebe98376a23fc78002c10f6 Jul 15 17:29:00.890133 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:29:00.890093 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" event={"ID":"db391423-e64a-4fbd-bed2-3b09f57ded38","Type":"ContainerStarted","Data":"86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd"} Jul 15 17:29:00.890297 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:29:00.890155 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" event={"ID":"db391423-e64a-4fbd-bed2-3b09f57ded38","Type":"ContainerStarted","Data":"eb4df14b1a1ed1b0f51623299752e499fb35e348cebe98376a23fc78002c10f6"} Jul 15 17:29:00.890297 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:29:00.890199 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:29:00.910948 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:29:00.910897 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" podStartSLOduration=2.46599785 podStartE2EDuration="2.910880116s" totalImagesPullingTime="444.882266ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:28:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:29:00.908946526 +0000 UTC m=+1791.150941766" watchObservedRunningTime="2026-07-15 17:29:00.910880116 +0000 UTC m=+1791.152875342" Jul 15 17:29:03.729043 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:29:03.729002 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:03.828969 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:29:03.828938 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:29:06.900954 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:29:06.900919 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:29:07.806407 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:29:07.806375 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:13.718209 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:13.718173 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn"] Jul 15 17:30:13.718777 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:13.718432 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" podUID="db391423-e64a-4fbd-bed2-3b09f57ded38" containerName="maas-api" containerID="cri-o://86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd" gracePeriod=30 Jul 15 17:30:13.765419 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:13.765392 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:13.971106 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:13.971043 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:30:14.065897 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.065868 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db391423-e64a-4fbd-bed2-3b09f57ded38-maas-api-tls\") pod \"db391423-e64a-4fbd-bed2-3b09f57ded38\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " Jul 15 17:30:14.066059 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.065977 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5ksf\" (UniqueName: \"kubernetes.io/projected/db391423-e64a-4fbd-bed2-3b09f57ded38-kube-api-access-d5ksf\") pod \"db391423-e64a-4fbd-bed2-3b09f57ded38\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " Jul 15 17:30:14.066059 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.066030 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db391423-e64a-4fbd-bed2-3b09f57ded38-openshift-service-ca\") pod \"db391423-e64a-4fbd-bed2-3b09f57ded38\" (UID: \"db391423-e64a-4fbd-bed2-3b09f57ded38\") " Jul 15 17:30:14.066435 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.066406 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db391423-e64a-4fbd-bed2-3b09f57ded38-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db391423-e64a-4fbd-bed2-3b09f57ded38" (UID: "db391423-e64a-4fbd-bed2-3b09f57ded38"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:30:14.068027 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.068004 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db391423-e64a-4fbd-bed2-3b09f57ded38-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db391423-e64a-4fbd-bed2-3b09f57ded38" (UID: "db391423-e64a-4fbd-bed2-3b09f57ded38"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:30:14.068151 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.068121 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db391423-e64a-4fbd-bed2-3b09f57ded38-kube-api-access-d5ksf" (OuterVolumeSpecName: "kube-api-access-d5ksf") pod "db391423-e64a-4fbd-bed2-3b09f57ded38" (UID: "db391423-e64a-4fbd-bed2-3b09f57ded38"). InnerVolumeSpecName "kube-api-access-d5ksf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:30:14.166924 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.166887 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db391423-e64a-4fbd-bed2-3b09f57ded38-openshift-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:30:14.166924 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.166915 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db391423-e64a-4fbd-bed2-3b09f57ded38-maas-api-tls\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:30:14.166924 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.166925 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5ksf\" (UniqueName: \"kubernetes.io/projected/db391423-e64a-4fbd-bed2-3b09f57ded38-kube-api-access-d5ksf\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:30:14.285868 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.285816 2562 generic.go:356] "Generic (PLEG): container finished" podID="db391423-e64a-4fbd-bed2-3b09f57ded38" containerID="86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd" exitCode=0 Jul 15 17:30:14.286022 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.285874 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" event={"ID":"db391423-e64a-4fbd-bed2-3b09f57ded38","Type":"ContainerDied","Data":"86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd"} Jul 15 17:30:14.286022 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.285898 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" Jul 15 17:30:14.286022 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.285938 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn" event={"ID":"db391423-e64a-4fbd-bed2-3b09f57ded38","Type":"ContainerDied","Data":"eb4df14b1a1ed1b0f51623299752e499fb35e348cebe98376a23fc78002c10f6"} Jul 15 17:30:14.286022 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.285969 2562 scope.go:117] "RemoveContainer" containerID="86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd" Jul 15 17:30:14.294988 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.294966 2562 scope.go:117] "RemoveContainer" containerID="86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd" Jul 15 17:30:14.295258 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:30:14.295235 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd\": container with ID starting with 86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd not found: ID does not exist" containerID="86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd" Jul 15 17:30:14.295353 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.295266 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd"} err="failed to get container status \"86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd\": rpc error: code = NotFound desc = could not find container \"86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd\": container with ID starting with 86b864d17f8566096619cc9c872d54e53d1eb5a7d11654ec50a31141742785bd not found: ID does not exist" Jul 15 17:30:14.305644 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.305622 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn"] Jul 15 17:30:14.309694 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:14.309663 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6feb2776-6c66bcc495-8mpnn"] Jul 15 17:30:16.251854 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:16.251807 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db391423-e64a-4fbd-bed2-3b09f57ded38" path="/var/lib/kubelet/pods/db391423-e64a-4fbd-bed2-3b09f57ded38/volumes" Jul 15 17:30:18.360277 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:18.360242 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:24.204658 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:24.204626 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:24.302983 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:24.302958 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:28.557603 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:28.557569 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:29.198341 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:29.198304 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:29.293053 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:29.293018 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:29.392495 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:29.392462 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:30.871724 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:30.871634 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:35.218453 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:35.218420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:36.815637 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:36.815584 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:36.929883 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:36.929824 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:37.016340 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:37.016310 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:37.116090 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:37.116058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:30:37.534532 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:30:37.534450 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:03.524377 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:03.524345 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:09.275409 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:09.275375 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:09.344121 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:09.344074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:09.427391 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:09.427357 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:09.525577 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:09.525496 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:25.386821 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:25.386776 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:31.132275 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:31.132237 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:31.273812 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:31.273784 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:31.375440 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:31.375413 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:41.891614 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:41.891580 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:42.706594 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:42.706545 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:48.398349 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:48.398309 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:48.459023 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:48.458987 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:48.541554 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:48.541518 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:48.645644 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:48.645609 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:48.744972 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:48.744891 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:49.469370 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:49.469326 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:49.565783 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:49.565748 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:49.666309 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:49.666280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:51.352584 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:51.352552 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:53.859501 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:53.859464 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:57.121093 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:57.121056 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:57.221487 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:57.221453 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:31:57.894182 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:31:57.894148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:03.539798 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:03.539707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:03.602488 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:03.602455 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:03.840086 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:03.840052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:04.193648 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:04.193559 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:04.509600 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:04.509513 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:07.164210 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.164173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:07.263848 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.263807 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:07.399523 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.399493 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4"] Jul 15 17:32:07.399932 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.399918 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db391423-e64a-4fbd-bed2-3b09f57ded38" containerName="maas-api" Jul 15 17:32:07.399988 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.399933 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="db391423-e64a-4fbd-bed2-3b09f57ded38" containerName="maas-api" Jul 15 17:32:07.400024 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.400008 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="db391423-e64a-4fbd-bed2-3b09f57ded38" containerName="maas-api" Jul 15 17:32:07.401499 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.401475 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:07.403122 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.403107 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:07.405679 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.405662 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4fc374e6-serving-cert\"" Jul 15 17:32:07.412955 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.412933 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4"] Jul 15 17:32:07.468078 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.467991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-maas-api-tls\") pod \"maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:07.468078 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.468031 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-openshift-service-ca\") pod \"maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:07.468264 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.468125 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vckb\" (UniqueName: \"kubernetes.io/projected/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-kube-api-access-6vckb\") pod \"maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:07.503149 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.503119 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:07.569469 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.569436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-maas-api-tls\") pod \"maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:07.569469 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.569477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-openshift-service-ca\") pod \"maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:07.569751 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.569510 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vckb\" (UniqueName: \"kubernetes.io/projected/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-kube-api-access-6vckb\") pod \"maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:07.569751 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:32:07.569591 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-serving-cert: secret "maas-api-e2e-mt-4fc374e6-serving-cert" not found Jul 15 17:32:07.569751 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:32:07.569675 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-maas-api-tls podName:9cb9223e-2045-4cfe-b74a-c7dbb21fa625 nodeName:}" failed. No retries permitted until 2026-07-15 17:32:08.069658578 +0000 UTC m=+1978.311653781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-maas-api-tls") pod "maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" (UID: "9cb9223e-2045-4cfe-b74a-c7dbb21fa625") : secret "maas-api-e2e-mt-4fc374e6-serving-cert" not found Jul 15 17:32:07.570266 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.570241 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-openshift-service-ca\") pod \"maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:07.578665 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.578643 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vckb\" (UniqueName: \"kubernetes.io/projected/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-kube-api-access-6vckb\") pod \"maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:07.669771 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.669735 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:07.950124 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:07.950087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:08.052932 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:08.052894 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:08.073642 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:08.073607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-maas-api-tls\") pod \"maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:08.076438 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:08.076407 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-maas-api-tls\") pod \"maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:08.315479 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:08.315394 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:08.671262 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:08.671231 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4"] Jul 15 17:32:08.672738 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:32:08.672707 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb9223e_2045_4cfe_b74a_c7dbb21fa625.slice/crio-63ad655c67e78717c5039d721cb80c4d923ab49c12467f9dd6ee37801d245946 WatchSource:0}: Error finding container 63ad655c67e78717c5039d721cb80c4d923ab49c12467f9dd6ee37801d245946: Status 404 returned error can't find the container with id 63ad655c67e78717c5039d721cb80c4d923ab49c12467f9dd6ee37801d245946 Jul 15 17:32:08.674187 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:08.674168 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:32:08.902606 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:08.902572 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" event={"ID":"9cb9223e-2045-4cfe-b74a-c7dbb21fa625","Type":"ContainerStarted","Data":"63ad655c67e78717c5039d721cb80c4d923ab49c12467f9dd6ee37801d245946"} Jul 15 17:32:09.909853 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:09.909799 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" event={"ID":"9cb9223e-2045-4cfe-b74a-c7dbb21fa625","Type":"ContainerStarted","Data":"b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be"} Jul 15 17:32:09.910254 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:09.909887 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:09.930610 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:09.930564 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" podStartSLOduration=2.256754437 podStartE2EDuration="2.930546482s" totalImagesPullingTime="673.792045ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:32:09.927709347 +0000 UTC m=+1980.169704586" watchObservedRunningTime="2026-07-15 17:32:09.930546482 +0000 UTC m=+1980.172541709" Jul 15 17:32:11.673086 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:11.673050 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:12.411510 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:12.411473 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:12.503539 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:12.503500 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:12.838815 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:12.838781 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:15.920658 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:15.920628 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:18.643471 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:18.643417 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:18.715770 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:18.715739 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:29.564507 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:29.564468 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4"] Jul 15 17:32:29.564929 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:29.564722 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" podUID="9cb9223e-2045-4cfe-b74a-c7dbb21fa625" containerName="maas-api" containerID="cri-o://b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be" gracePeriod=30 Jul 15 17:32:29.606918 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:29.606870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:29.821211 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:29.821147 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:29.979968 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:29.979940 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vckb\" (UniqueName: \"kubernetes.io/projected/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-kube-api-access-6vckb\") pod \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " Jul 15 17:32:29.980170 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:29.979983 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-maas-api-tls\") pod \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " Jul 15 17:32:29.980170 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:29.980015 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-openshift-service-ca\") pod \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\" (UID: \"9cb9223e-2045-4cfe-b74a-c7dbb21fa625\") " Jul 15 17:32:29.980381 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:29.980357 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9cb9223e-2045-4cfe-b74a-c7dbb21fa625" (UID: "9cb9223e-2045-4cfe-b74a-c7dbb21fa625"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:32:29.982231 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:29.982206 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9cb9223e-2045-4cfe-b74a-c7dbb21fa625" (UID: "9cb9223e-2045-4cfe-b74a-c7dbb21fa625"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:32:29.982348 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:29.982329 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-kube-api-access-6vckb" (OuterVolumeSpecName: "kube-api-access-6vckb") pod "9cb9223e-2045-4cfe-b74a-c7dbb21fa625" (UID: "9cb9223e-2045-4cfe-b74a-c7dbb21fa625"). InnerVolumeSpecName "kube-api-access-6vckb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:32:30.027922 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.027892 2562 generic.go:356] "Generic (PLEG): container finished" podID="9cb9223e-2045-4cfe-b74a-c7dbb21fa625" containerID="b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be" exitCode=0 Jul 15 17:32:30.028082 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.027948 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" Jul 15 17:32:30.028082 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.027983 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" event={"ID":"9cb9223e-2045-4cfe-b74a-c7dbb21fa625","Type":"ContainerDied","Data":"b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be"} Jul 15 17:32:30.028082 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.028038 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4" event={"ID":"9cb9223e-2045-4cfe-b74a-c7dbb21fa625","Type":"ContainerDied","Data":"63ad655c67e78717c5039d721cb80c4d923ab49c12467f9dd6ee37801d245946"} Jul 15 17:32:30.028082 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.028068 2562 scope.go:117] "RemoveContainer" containerID="b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be" Jul 15 17:32:30.038406 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.038385 2562 scope.go:117] "RemoveContainer" containerID="b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be" Jul 15 17:32:30.038691 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:32:30.038666 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be\": container with ID starting with b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be not found: ID does not exist" containerID="b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be" Jul 15 17:32:30.038747 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.038699 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be"} err="failed to get container status \"b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be\": rpc error: code = NotFound desc = could not find container \"b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be\": container with ID starting with b20a222634ccb283ae613c0931bb4b04cd4deec4e9b4fefef469b05fdbe3c3be not found: ID does not exist" Jul 15 17:32:30.051441 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.051415 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4"] Jul 15 17:32:30.055568 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.055547 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4fc374e6-664c5bf8bd-khsq4"] Jul 15 17:32:30.081547 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.081523 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-openshift-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:32:30.081547 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.081547 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vckb\" (UniqueName: \"kubernetes.io/projected/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-kube-api-access-6vckb\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:32:30.081702 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.081557 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb9223e-2045-4cfe-b74a-c7dbb21fa625-maas-api-tls\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:32:30.258498 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:30.258461 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cb9223e-2045-4cfe-b74a-c7dbb21fa625" path="/var/lib/kubelet/pods/9cb9223e-2045-4cfe-b74a-c7dbb21fa625/volumes" Jul 15 17:32:34.096388 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:34.096361 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:39.073820 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:39.073776 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:39.823893 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:39.823828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:39.891323 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:39.891283 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:44.777466 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:44.777430 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:45.003035 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:45.002991 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:45.220696 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:45.220652 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:45.326171 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:45.326132 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:45.422762 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:45.422728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:51.128737 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:51.128707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:51.250536 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:51.250500 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:51.389740 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:51.389647 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:51.510494 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:51.510445 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:57.286914 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:57.286885 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:32:57.353425 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:32:57.353395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:04.113209 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:04.113153 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:04.506947 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:04.506864 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:04.679361 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:04.679307 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:07.418567 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.418536 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:07.529163 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.529128 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:07.618598 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.618566 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh"] Jul 15 17:33:07.619178 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.619163 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cb9223e-2045-4cfe-b74a-c7dbb21fa625" containerName="maas-api" Jul 15 17:33:07.619231 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.619182 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cb9223e-2045-4cfe-b74a-c7dbb21fa625" containerName="maas-api" Jul 15 17:33:07.619283 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.619273 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cb9223e-2045-4cfe-b74a-c7dbb21fa625" containerName="maas-api" Jul 15 17:33:07.622266 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.622245 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:07.625162 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.625140 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6a6deab7-serving-cert\"" Jul 15 17:33:07.635891 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.635868 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh"] Jul 15 17:33:07.668923 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.668853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:07.726431 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.726406 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eabdcb36-ddef-4c61-a810-00c1a306262f-maas-api-tls\") pod \"maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:07.726563 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.726463 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk8xm\" (UniqueName: \"kubernetes.io/projected/eabdcb36-ddef-4c61-a810-00c1a306262f-kube-api-access-kk8xm\") pod \"maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:07.726563 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.726537 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eabdcb36-ddef-4c61-a810-00c1a306262f-openshift-service-ca\") pod \"maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:07.793550 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.793510 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:07.827715 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.827688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk8xm\" (UniqueName: \"kubernetes.io/projected/eabdcb36-ddef-4c61-a810-00c1a306262f-kube-api-access-kk8xm\") pod \"maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:07.827882 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.827732 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eabdcb36-ddef-4c61-a810-00c1a306262f-openshift-service-ca\") pod \"maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:07.827882 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.827780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eabdcb36-ddef-4c61-a810-00c1a306262f-maas-api-tls\") pod \"maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:07.828382 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.828358 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eabdcb36-ddef-4c61-a810-00c1a306262f-openshift-service-ca\") pod \"maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:07.830327 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.830308 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eabdcb36-ddef-4c61-a810-00c1a306262f-maas-api-tls\") pod \"maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:07.841989 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.841969 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk8xm\" (UniqueName: \"kubernetes.io/projected/eabdcb36-ddef-4c61-a810-00c1a306262f-kube-api-access-kk8xm\") pod \"maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:07.935765 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:07.935703 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:08.101396 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:08.101339 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh"] Jul 15 17:33:08.101911 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:33:08.101883 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeabdcb36_ddef_4c61_a810_00c1a306262f.slice/crio-0722736b6d6a2200aa2a35f4a7baa2b866e73c4d81776304ce2346a7e8911b16 WatchSource:0}: Error finding container 0722736b6d6a2200aa2a35f4a7baa2b866e73c4d81776304ce2346a7e8911b16: Status 404 returned error can't find the container with id 0722736b6d6a2200aa2a35f4a7baa2b866e73c4d81776304ce2346a7e8911b16 Jul 15 17:33:08.243335 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:08.243252 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" event={"ID":"eabdcb36-ddef-4c61-a810-00c1a306262f","Type":"ContainerStarted","Data":"0722736b6d6a2200aa2a35f4a7baa2b866e73c4d81776304ce2346a7e8911b16"} Jul 15 17:33:08.283038 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:08.283011 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:08.387763 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:08.387732 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:09.250680 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:09.250643 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" event={"ID":"eabdcb36-ddef-4c61-a810-00c1a306262f","Type":"ContainerStarted","Data":"dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d"} Jul 15 17:33:09.251085 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:09.250754 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:09.273063 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:09.273015 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" podStartSLOduration=1.8382086850000001 podStartE2EDuration="2.272999919s" totalImagesPullingTime="434.791234ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:33:09.270515984 +0000 UTC m=+2039.512511203" watchObservedRunningTime="2026-07-15 17:33:09.272999919 +0000 UTC m=+2039.514995145" Jul 15 17:33:10.369723 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:10.369694 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:10.472113 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:10.472079 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:11.900341 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:11.900311 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:15.261857 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:15.261813 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:33:23.198378 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:23.198336 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:29.151818 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:29.151779 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:33:29.248093 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:33:29.248052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:00.705649 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:00.705451 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh"] Jul 15 17:34:00.706218 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:00.705923 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" podUID="eabdcb36-ddef-4c61-a810-00c1a306262f" containerName="maas-api" containerID="cri-o://dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d" gracePeriod=30 Jul 15 17:34:00.722096 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:00.722064 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:00.954766 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:00.954740 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:34:01.012911 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.012798 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eabdcb36-ddef-4c61-a810-00c1a306262f-openshift-service-ca\") pod \"eabdcb36-ddef-4c61-a810-00c1a306262f\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " Jul 15 17:34:01.012911 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.012891 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk8xm\" (UniqueName: \"kubernetes.io/projected/eabdcb36-ddef-4c61-a810-00c1a306262f-kube-api-access-kk8xm\") pod \"eabdcb36-ddef-4c61-a810-00c1a306262f\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " Jul 15 17:34:01.012911 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.012918 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eabdcb36-ddef-4c61-a810-00c1a306262f-maas-api-tls\") pod \"eabdcb36-ddef-4c61-a810-00c1a306262f\" (UID: \"eabdcb36-ddef-4c61-a810-00c1a306262f\") " Jul 15 17:34:01.013219 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.013186 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eabdcb36-ddef-4c61-a810-00c1a306262f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eabdcb36-ddef-4c61-a810-00c1a306262f" (UID: "eabdcb36-ddef-4c61-a810-00c1a306262f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:34:01.015270 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.015239 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eabdcb36-ddef-4c61-a810-00c1a306262f-kube-api-access-kk8xm" (OuterVolumeSpecName: "kube-api-access-kk8xm") pod "eabdcb36-ddef-4c61-a810-00c1a306262f" (UID: "eabdcb36-ddef-4c61-a810-00c1a306262f"). InnerVolumeSpecName "kube-api-access-kk8xm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:34:01.015270 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.015257 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eabdcb36-ddef-4c61-a810-00c1a306262f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eabdcb36-ddef-4c61-a810-00c1a306262f" (UID: "eabdcb36-ddef-4c61-a810-00c1a306262f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:34:01.113449 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.113414 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eabdcb36-ddef-4c61-a810-00c1a306262f-openshift-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:34:01.113449 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.113441 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk8xm\" (UniqueName: \"kubernetes.io/projected/eabdcb36-ddef-4c61-a810-00c1a306262f-kube-api-access-kk8xm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:34:01.113449 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.113453 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eabdcb36-ddef-4c61-a810-00c1a306262f-maas-api-tls\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:34:01.553424 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.553392 2562 generic.go:356] "Generic (PLEG): container finished" podID="eabdcb36-ddef-4c61-a810-00c1a306262f" containerID="dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d" exitCode=0 Jul 15 17:34:01.553599 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.553434 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" Jul 15 17:34:01.553599 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.553481 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" event={"ID":"eabdcb36-ddef-4c61-a810-00c1a306262f","Type":"ContainerDied","Data":"dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d"} Jul 15 17:34:01.553599 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.553535 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh" event={"ID":"eabdcb36-ddef-4c61-a810-00c1a306262f","Type":"ContainerDied","Data":"0722736b6d6a2200aa2a35f4a7baa2b866e73c4d81776304ce2346a7e8911b16"} Jul 15 17:34:01.553599 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.553560 2562 scope.go:117] "RemoveContainer" containerID="dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d" Jul 15 17:34:01.562933 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.562916 2562 scope.go:117] "RemoveContainer" containerID="dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d" Jul 15 17:34:01.563209 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:34:01.563186 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d\": container with ID starting with dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d not found: ID does not exist" containerID="dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d" Jul 15 17:34:01.563263 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.563221 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d"} err="failed to get container status \"dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d\": rpc error: code = NotFound desc = could not find container \"dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d\": container with ID starting with dae307d9c710e7b9743db869be439126d4b316ffd1ebd75003990585b041828d not found: ID does not exist" Jul 15 17:34:01.575489 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.575461 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh"] Jul 15 17:34:01.579617 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:01.579593 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a6deab7-68bc96c4f8-spwmh"] Jul 15 17:34:02.251100 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:02.251073 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eabdcb36-ddef-4c61-a810-00c1a306262f" path="/var/lib/kubelet/pods/eabdcb36-ddef-4c61-a810-00c1a306262f/volumes" Jul 15 17:34:05.306057 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:05.306012 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:10.388049 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:10.387999 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:41.803951 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:41.803914 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:42.923613 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:42.923579 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:48.433053 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:48.433005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:48.536427 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:48.536391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:57.608754 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:57.608719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:57.844363 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:57.844327 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:58.887444 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:58.887413 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:59.069851 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:59.069813 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:59.224770 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:59.224683 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:59.914816 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:59.914782 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:34:59.998031 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:34:59.998002 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:00.098738 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:00.098705 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:01.480044 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:01.479967 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:07.185255 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:07.185216 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:07.307934 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:07.307904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:07.859321 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:07.859281 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:13.905097 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:13.905059 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:13.976813 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:13.976780 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:14.069083 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:14.069047 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:14.170773 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:14.170671 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:24.606789 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:24.606750 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:24.952818 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:24.952720 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:45.948305 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:45.948263 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:51.886480 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:51.886440 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:51.950021 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:51.949986 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:52.140646 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:52.140550 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:52.462087 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:52.462006 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:52.822785 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:52.822746 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:55.286033 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.285992 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:55.369641 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.369607 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:55.370256 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.370232 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh"] Jul 15 17:35:55.370611 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.370600 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eabdcb36-ddef-4c61-a810-00c1a306262f" containerName="maas-api" Jul 15 17:35:55.370655 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.370613 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eabdcb36-ddef-4c61-a810-00c1a306262f" containerName="maas-api" Jul 15 17:35:55.370698 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.370689 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="eabdcb36-ddef-4c61-a810-00c1a306262f" containerName="maas-api" Jul 15 17:35:55.373780 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.373760 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:55.376406 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.376385 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a6dd94-serving-cert\"" Jul 15 17:35:55.389617 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.389589 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh"] Jul 15 17:35:55.452803 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.452751 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz785\" (UniqueName: \"kubernetes.io/projected/96bf5737-be31-47f4-8907-1f5f24c7c137-kube-api-access-wz785\") pod \"maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:55.453034 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.452856 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96bf5737-be31-47f4-8907-1f5f24c7c137-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:55.453034 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.452906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls\") pod \"maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:55.523994 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.523960 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:55.554557 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.554462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96bf5737-be31-47f4-8907-1f5f24c7c137-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:55.554557 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.554510 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls\") pod \"maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:55.554802 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.554575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz785\" (UniqueName: \"kubernetes.io/projected/96bf5737-be31-47f4-8907-1f5f24c7c137-kube-api-access-wz785\") pod \"maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:55.554802 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:35:55.554687 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-serving-cert: secret "maas-api-e2e-inf-a-a6dd94-serving-cert" not found Jul 15 17:35:55.554802 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:35:55.554761 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls podName:96bf5737-be31-47f4-8907-1f5f24c7c137 nodeName:}" failed. No retries permitted until 2026-07-15 17:35:56.054744646 +0000 UTC m=+2206.296739851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls") pod "maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" (UID: "96bf5737-be31-47f4-8907-1f5f24c7c137") : secret "maas-api-e2e-inf-a-a6dd94-serving-cert" not found Jul 15 17:35:55.555259 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.555235 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96bf5737-be31-47f4-8907-1f5f24c7c137-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:55.574703 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:55.574667 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz785\" (UniqueName: \"kubernetes.io/projected/96bf5737-be31-47f4-8907-1f5f24c7c137-kube-api-access-wz785\") pod \"maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:56.059680 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:56.059638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls\") pod \"maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:56.059885 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:35:56.059769 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-serving-cert: secret "maas-api-e2e-inf-a-a6dd94-serving-cert" not found Jul 15 17:35:56.059885 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:35:56.059849 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls podName:96bf5737-be31-47f4-8907-1f5f24c7c137 nodeName:}" failed. No retries permitted until 2026-07-15 17:35:57.059817259 +0000 UTC m=+2207.301812464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls") pod "maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" (UID: "96bf5737-be31-47f4-8907-1f5f24c7c137") : secret "maas-api-e2e-inf-a-a6dd94-serving-cert" not found Jul 15 17:35:56.065053 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:56.065021 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:56.165864 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:56.165798 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:35:57.067565 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:57.067531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls\") pod \"maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:57.070315 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:57.070286 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls\") pod \"maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:57.183385 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:57.183354 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:57.536678 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:57.536644 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh"] Jul 15 17:35:57.538481 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:35:57.538449 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96bf5737_be31_47f4_8907_1f5f24c7c137.slice/crio-86d66c502223eae249ffe97ff1c6657790c0eef07460516f4ac67ba1f4c96319 WatchSource:0}: Error finding container 86d66c502223eae249ffe97ff1c6657790c0eef07460516f4ac67ba1f4c96319: Status 404 returned error can't find the container with id 86d66c502223eae249ffe97ff1c6657790c0eef07460516f4ac67ba1f4c96319 Jul 15 17:35:58.192717 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:58.192690 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" event={"ID":"96bf5737-be31-47f4-8907-1f5f24c7c137","Type":"ContainerStarted","Data":"98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b"} Jul 15 17:35:58.193026 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:58.192737 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" event={"ID":"96bf5737-be31-47f4-8907-1f5f24c7c137","Type":"ContainerStarted","Data":"86d66c502223eae249ffe97ff1c6657790c0eef07460516f4ac67ba1f4c96319"} Jul 15 17:35:58.193026 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:58.192814 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:35:58.212707 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:58.212663 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" podStartSLOduration=2.661928804 podStartE2EDuration="3.212648263s" totalImagesPullingTime="550.719459ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:35:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:35:58.210895902 +0000 UTC m=+2208.452891141" watchObservedRunningTime="2026-07-15 17:35:58.212648263 +0000 UTC m=+2208.454643489" Jul 15 17:35:59.774163 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:35:59.774119 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:00.097873 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:00.097799 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:00.411603 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:00.411521 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:00.512318 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:00.512285 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:03.310075 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:03.310031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:03.413029 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:03.412976 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:03.583987 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:03.583954 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:03.936558 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:03.936463 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:04.125381 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:04.125341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:04.206697 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:04.206618 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:36:04.226599 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:04.226563 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:07.842880 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:07.842824 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:17.970830 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:17.970794 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:18.071125 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:18.071094 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:19.152468 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:19.152363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:25.040479 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:25.040447 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:25.143177 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:25.143141 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:36:44.336222 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:36:44.336106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:16.737496 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:16.737444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:17.625462 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:17.625430 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:18.843334 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:18.843292 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:18.940752 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:18.940715 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:28.374806 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.374773 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh"] Jul 15 17:37:28.375288 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.375032 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" podUID="96bf5737-be31-47f4-8907-1f5f24c7c137" containerName="maas-api" containerID="cri-o://98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b" gracePeriod=30 Jul 15 17:37:28.410982 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.410953 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:28.626688 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.626638 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:37:28.700281 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.700248 2562 generic.go:356] "Generic (PLEG): container finished" podID="96bf5737-be31-47f4-8907-1f5f24c7c137" containerID="98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b" exitCode=0 Jul 15 17:37:28.700430 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.700294 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" Jul 15 17:37:28.700430 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.700302 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" event={"ID":"96bf5737-be31-47f4-8907-1f5f24c7c137","Type":"ContainerDied","Data":"98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b"} Jul 15 17:37:28.700430 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.700345 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh" event={"ID":"96bf5737-be31-47f4-8907-1f5f24c7c137","Type":"ContainerDied","Data":"86d66c502223eae249ffe97ff1c6657790c0eef07460516f4ac67ba1f4c96319"} Jul 15 17:37:28.700430 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.700372 2562 scope.go:117] "RemoveContainer" containerID="98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b" Jul 15 17:37:28.709081 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.709060 2562 scope.go:117] "RemoveContainer" containerID="98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b" Jul 15 17:37:28.709348 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:37:28.709331 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b\": container with ID starting with 98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b not found: ID does not exist" containerID="98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b" Jul 15 17:37:28.709401 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.709357 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b"} err="failed to get container status \"98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b\": rpc error: code = NotFound desc = could not find container \"98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b\": container with ID starting with 98ea46f3f876991148bec3e27cb1f3fb2aab7cc8ed5caa661d18ce2860ab1c3b not found: ID does not exist" Jul 15 17:37:28.798270 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.798232 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96bf5737-be31-47f4-8907-1f5f24c7c137-openshift-service-ca\") pod \"96bf5737-be31-47f4-8907-1f5f24c7c137\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " Jul 15 17:37:28.798431 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.798308 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls\") pod \"96bf5737-be31-47f4-8907-1f5f24c7c137\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " Jul 15 17:37:28.798431 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.798371 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wz785\" (UniqueName: \"kubernetes.io/projected/96bf5737-be31-47f4-8907-1f5f24c7c137-kube-api-access-wz785\") pod \"96bf5737-be31-47f4-8907-1f5f24c7c137\" (UID: \"96bf5737-be31-47f4-8907-1f5f24c7c137\") " Jul 15 17:37:28.798642 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.798614 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96bf5737-be31-47f4-8907-1f5f24c7c137-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "96bf5737-be31-47f4-8907-1f5f24c7c137" (UID: "96bf5737-be31-47f4-8907-1f5f24c7c137"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:37:28.798743 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.798680 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96bf5737-be31-47f4-8907-1f5f24c7c137-openshift-service-ca\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:37:28.800413 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.800391 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "96bf5737-be31-47f4-8907-1f5f24c7c137" (UID: "96bf5737-be31-47f4-8907-1f5f24c7c137"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:37:28.800489 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.800451 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96bf5737-be31-47f4-8907-1f5f24c7c137-kube-api-access-wz785" (OuterVolumeSpecName: "kube-api-access-wz785") pod "96bf5737-be31-47f4-8907-1f5f24c7c137" (UID: "96bf5737-be31-47f4-8907-1f5f24c7c137"). InnerVolumeSpecName "kube-api-access-wz785". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:37:28.899790 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.899704 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96bf5737-be31-47f4-8907-1f5f24c7c137-maas-api-tls\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:37:28.899790 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:28.899736 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz785\" (UniqueName: \"kubernetes.io/projected/96bf5737-be31-47f4-8907-1f5f24c7c137-kube-api-access-wz785\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:37:29.024141 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:29.024112 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh"] Jul 15 17:37:29.028453 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:29.028429 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a6dd94-f8d5b7bc4-tvkjh"] Jul 15 17:37:30.254297 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:30.254257 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96bf5737-be31-47f4-8907-1f5f24c7c137" path="/var/lib/kubelet/pods/96bf5737-be31-47f4-8907-1f5f24c7c137/volumes" Jul 15 17:37:32.866016 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:32.865986 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:38.804243 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:38.804205 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:38.904128 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:38.904097 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:43.229753 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:43.229718 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:43.570017 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:43.569921 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:43.812202 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:43.812168 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:44.538692 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:44.538648 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:44.985171 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:44.985133 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:54.361542 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:54.361501 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:54.696119 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:54.696031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:54.754513 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:54.754479 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:37:59.026408 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:37:59.026375 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:05.276643 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:05.276555 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:05.343305 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:05.343276 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:09.758970 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:09.758938 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:09.977157 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:09.977123 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:10.330474 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:10.330443 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:10.645157 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:10.645068 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:10.856452 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:10.856420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:11.423608 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:11.423564 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:11.521388 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:11.521355 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:14.077118 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:14.077074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:14.177375 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:14.177339 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:14.316570 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:14.316534 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:14.894830 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:14.894792 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:15.880592 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:15.880557 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:19.393887 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:19.393853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:19.498001 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:19.497970 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:19.660143 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:19.660049 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:22.071822 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:22.071785 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:22.152314 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:22.152275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:22.250075 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:22.250039 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:22.673504 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:22.673468 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:22.774156 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:22.774116 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:22.836229 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:22.836196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:26.662733 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:26.662689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:35.355823 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:35.355761 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:35.454721 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:35.454688 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:37.080185 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:37.080139 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:43.677829 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:43.677788 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:43.839259 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:43.839224 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:45.684326 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:45.684286 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:45.786373 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:45.786334 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:48.415460 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:48.415427 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:49.878099 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:49.878058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:55.900985 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:55.900951 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:38:55.968306 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:38:55.968268 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:00.565074 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:00.565032 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:00.802064 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:00.802031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:01.466507 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:01.466467 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:01.567016 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:01.566983 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:11.609263 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:11.609228 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:16.033335 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:16.033299 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:21.965802 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:21.965773 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:22.027040 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:22.027002 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:26.741795 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:26.741761 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:27.000858 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:27.000724 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:27.352029 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:27.351991 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:27.705421 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:27.705340 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:27.815849 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:27.815802 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:30.427956 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:30.427903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:30.523555 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:30.523479 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:30.597113 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:30.597079 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:30.710386 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:30.710277 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:31.137752 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:31.137721 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:31.238052 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:31.238022 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:35.012321 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:35.012287 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:35.314797 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:35.314711 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:35.648475 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:35.648449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:35.896624 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:35.896590 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:38.468345 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:38.468304 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:38.583872 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:38.583820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:38.772180 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:38.772093 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:38.909678 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:38.909651 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:39.289723 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:39.289687 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:39.390571 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:39.390540 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:42.962031 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:42.961995 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:51.917651 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:51.917604 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:52.259975 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:52.259886 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:39:53.082513 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:39:53.082476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:00.170261 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:00.170229 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:00.271495 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:00.271453 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:09.913556 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:09.913522 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh"] Jul 15 17:40:09.914142 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:09.914126 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96bf5737-be31-47f4-8907-1f5f24c7c137" containerName="maas-api" Jul 15 17:40:09.914242 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:09.914145 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="96bf5737-be31-47f4-8907-1f5f24c7c137" containerName="maas-api" Jul 15 17:40:09.914242 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:09.914220 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="96bf5737-be31-47f4-8907-1f5f24c7c137" containerName="maas-api" Jul 15 17:40:09.917273 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:09.917257 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:09.921212 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:09.921177 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fadd3e\"/\"kube-root-ca.crt\"" Jul 15 17:40:09.921355 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:09.921269 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fadd3e\"/\"default-dockercfg-xtz49\"" Jul 15 17:40:09.921355 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:09.921292 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fadd3e\"/\"auth-test-model-fadd3e-kserve-self-signed-certs\"" Jul 15 17:40:09.921558 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:09.921544 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fadd3e\"/\"openshift-service-ca.crt\"" Jul 15 17:40:09.927374 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:09.927350 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh"] Jul 15 17:40:09.942053 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:09.942030 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:40:10.014705 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.014670 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tmp-dir\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.014917 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.014717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tls-certs\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.014917 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.014811 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-home\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.014917 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.014873 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-dshm\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.014917 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.014897 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn8xx\" (UniqueName: \"kubernetes.io/projected/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-kube-api-access-bn8xx\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.015106 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.014932 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-model-cache\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.115813 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.115784 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tmp-dir\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.116006 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.115824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tls-certs\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.116006 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.115877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-home\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.116006 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.115895 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-dshm\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.116006 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.115914 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn8xx\" (UniqueName: \"kubernetes.io/projected/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-kube-api-access-bn8xx\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.116223 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.116044 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-model-cache\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.116223 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.116188 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tmp-dir\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.116325 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.116270 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-home\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.116363 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.116348 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-model-cache\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.118202 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.118185 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-dshm\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.119044 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.119029 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fadd3e\"/\"auth-test-model-fadd3e-kserve-self-signed-certs\"" Jul 15 17:40:10.125793 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.125775 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fadd3e\"/\"kube-root-ca.crt\"" Jul 15 17:40:10.129027 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.129006 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tls-certs\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.136475 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.136457 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fadd3e\"/\"openshift-service-ca.crt\"" Jul 15 17:40:10.146489 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.146462 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn8xx\" (UniqueName: \"kubernetes.io/projected/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-kube-api-access-bn8xx\") pod \"auth-test-model-fadd3e-kserve-84d897d759-whxwh\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.231415 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.231349 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fadd3e\"/\"default-dockercfg-xtz49\"" Jul 15 17:40:10.239081 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.239062 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:10.380306 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.380280 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh"] Jul 15 17:40:10.381919 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:40:10.381889 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99eb1e7f_0cbc_4e16_8b57_2481ae447f01.slice/crio-ea6728792a4b0b8cf1c730511d616f6ad8a83cbd04dc2dcecaaefce6aeed8bef WatchSource:0}: Error finding container ea6728792a4b0b8cf1c730511d616f6ad8a83cbd04dc2dcecaaefce6aeed8bef: Status 404 returned error can't find the container with id ea6728792a4b0b8cf1c730511d616f6ad8a83cbd04dc2dcecaaefce6aeed8bef Jul 15 17:40:10.598520 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.598487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" event={"ID":"99eb1e7f-0cbc-4e16-8b57-2481ae447f01","Type":"ContainerStarted","Data":"20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4"} Jul 15 17:40:10.598714 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.598529 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" event={"ID":"99eb1e7f-0cbc-4e16-8b57-2481ae447f01","Type":"ContainerStarted","Data":"ea6728792a4b0b8cf1c730511d616f6ad8a83cbd04dc2dcecaaefce6aeed8bef"} Jul 15 17:40:10.619915 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:10.619870 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" podStartSLOduration=1.6198527120000001 podStartE2EDuration="1.619852712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:40:10.618231141 +0000 UTC m=+2460.860226369" watchObservedRunningTime="2026-07-15 17:40:10.619852712 +0000 UTC m=+2460.861847937" Jul 15 17:40:20.239621 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:20.239581 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:20.239621 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:20.239631 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:20.252588 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:20.252564 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:20.663315 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:20.663284 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:40:38.637944 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:40:38.637905 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:41:08.248879 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:41:08.248829 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:41:10.456624 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:41:10.456589 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:41:25.224272 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:41:25.224239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:42:09.877001 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:09.876960 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:42:10.728260 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:10.728226 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh"] Jul 15 17:42:10.774606 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:10.774578 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:42:25.748668 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:25.748608 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" podUID="99eb1e7f-0cbc-4e16-8b57-2481ae447f01" containerName="main" containerID="cri-o://20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4" gracePeriod=45 Jul 15 17:42:26.007796 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.007732 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:42:26.026281 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.026257 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn8xx\" (UniqueName: \"kubernetes.io/projected/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-kube-api-access-bn8xx\") pod \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " Jul 15 17:42:26.026398 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.026293 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-model-cache\") pod \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " Jul 15 17:42:26.026398 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.026329 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-home\") pod \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " Jul 15 17:42:26.026398 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.026359 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tls-certs\") pod \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " Jul 15 17:42:26.026398 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.026382 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-dshm\") pod \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " Jul 15 17:42:26.026609 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.026400 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tmp-dir\") pod \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\" (UID: \"99eb1e7f-0cbc-4e16-8b57-2481ae447f01\") " Jul 15 17:42:26.026663 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.026621 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-model-cache" (OuterVolumeSpecName: "model-cache") pod "99eb1e7f-0cbc-4e16-8b57-2481ae447f01" (UID: "99eb1e7f-0cbc-4e16-8b57-2481ae447f01"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:42:26.026663 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.026636 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-home" (OuterVolumeSpecName: "home") pod "99eb1e7f-0cbc-4e16-8b57-2481ae447f01" (UID: "99eb1e7f-0cbc-4e16-8b57-2481ae447f01"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:42:26.026891 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.026867 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "99eb1e7f-0cbc-4e16-8b57-2481ae447f01" (UID: "99eb1e7f-0cbc-4e16-8b57-2481ae447f01"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:42:26.028607 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.028583 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-kube-api-access-bn8xx" (OuterVolumeSpecName: "kube-api-access-bn8xx") pod "99eb1e7f-0cbc-4e16-8b57-2481ae447f01" (UID: "99eb1e7f-0cbc-4e16-8b57-2481ae447f01"). InnerVolumeSpecName "kube-api-access-bn8xx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:42:26.028687 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.028632 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "99eb1e7f-0cbc-4e16-8b57-2481ae447f01" (UID: "99eb1e7f-0cbc-4e16-8b57-2481ae447f01"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:42:26.028687 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.028646 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-dshm" (OuterVolumeSpecName: "dshm") pod "99eb1e7f-0cbc-4e16-8b57-2481ae447f01" (UID: "99eb1e7f-0cbc-4e16-8b57-2481ae447f01"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:42:26.127414 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.127386 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn8xx\" (UniqueName: \"kubernetes.io/projected/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-kube-api-access-bn8xx\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:42:26.127414 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.127411 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:42:26.127414 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.127420 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:42:26.127610 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.127428 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:42:26.127610 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.127435 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:42:26.127610 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.127442 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99eb1e7f-0cbc-4e16-8b57-2481ae447f01-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:42:26.335701 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.335671 2562 generic.go:356] "Generic (PLEG): container finished" podID="99eb1e7f-0cbc-4e16-8b57-2481ae447f01" containerID="20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4" exitCode=0 Jul 15 17:42:26.335850 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.335735 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" Jul 15 17:42:26.335850 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.335754 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" event={"ID":"99eb1e7f-0cbc-4e16-8b57-2481ae447f01","Type":"ContainerDied","Data":"20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4"} Jul 15 17:42:26.335850 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.335793 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh" event={"ID":"99eb1e7f-0cbc-4e16-8b57-2481ae447f01","Type":"ContainerDied","Data":"ea6728792a4b0b8cf1c730511d616f6ad8a83cbd04dc2dcecaaefce6aeed8bef"} Jul 15 17:42:26.335850 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.335810 2562 scope.go:117] "RemoveContainer" containerID="20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4" Jul 15 17:42:26.345309 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.345291 2562 scope.go:117] "RemoveContainer" containerID="20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4" Jul 15 17:42:26.345558 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:42:26.345537 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4\": container with ID starting with 20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4 not found: ID does not exist" containerID="20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4" Jul 15 17:42:26.345614 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.345566 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4"} err="failed to get container status \"20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4\": rpc error: code = NotFound desc = could not find container \"20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4\": container with ID starting with 20900f1bcfd558e7c623d4888f065c2d7eb408545e7640357135fb43d04e66f4 not found: ID does not exist" Jul 15 17:42:26.352855 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.352757 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh"] Jul 15 17:42:26.355417 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.355397 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fadd3e/auth-test-model-fadd3e-kserve-84d897d759-whxwh"] Jul 15 17:42:26.746937 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.743977 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2"] Jul 15 17:42:26.746937 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.744546 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99eb1e7f-0cbc-4e16-8b57-2481ae447f01" containerName="main" Jul 15 17:42:26.746937 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.744564 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="99eb1e7f-0cbc-4e16-8b57-2481ae447f01" containerName="main" Jul 15 17:42:26.746937 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.744672 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="99eb1e7f-0cbc-4e16-8b57-2481ae447f01" containerName="main" Jul 15 17:42:26.749574 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.749542 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.752755 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.752736 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0bc296\"/\"kube-root-ca.crt\"" Jul 15 17:42:26.753243 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.753226 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0bc296\"/\"openshift-service-ca.crt\"" Jul 15 17:42:26.753947 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.753928 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0bc296\"/\"default-dockercfg-tds97\"" Jul 15 17:42:26.754040 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.753933 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0bc296\"/\"sub-test-model-0bc296-kserve-self-signed-certs\"" Jul 15 17:42:26.761041 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.761019 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2"] Jul 15 17:42:26.769213 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.769183 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:42:26.833296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.833262 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tmp-dir\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.833296 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.833304 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tls-certs\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.833524 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.833328 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcvrh\" (UniqueName: \"kubernetes.io/projected/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-kube-api-access-tcvrh\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.833524 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.833352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-model-cache\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.833524 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.833453 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-home\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.833524 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.833479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-dshm\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.934656 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.934624 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-home\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.934656 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.934659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-dshm\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.934945 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.934723 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tmp-dir\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.934945 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.934746 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tls-certs\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.934945 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.934783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcvrh\" (UniqueName: \"kubernetes.io/projected/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-kube-api-access-tcvrh\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.934945 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.934811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-model-cache\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.935164 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.935104 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-home\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.935164 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.935127 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tmp-dir\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.935356 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.935332 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-model-cache\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.937316 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.937285 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-dshm\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.937439 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.937415 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tls-certs\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:26.948445 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:26.948414 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcvrh\" (UniqueName: \"kubernetes.io/projected/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-kube-api-access-tcvrh\") pod \"sub-test-model-0bc296-kserve-68865c5b7-n2tz2\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:27.060317 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:27.060228 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:27.210946 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:27.210911 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2"] Jul 15 17:42:27.211817 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:42:27.211785 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32dc4ff3_62f3_401a_a8da_fe5fbd24dc0b.slice/crio-93345b0d9b62233050abb71cab4f21327dcb3dab36fbfdc2fcefb68975254631 WatchSource:0}: Error finding container 93345b0d9b62233050abb71cab4f21327dcb3dab36fbfdc2fcefb68975254631: Status 404 returned error can't find the container with id 93345b0d9b62233050abb71cab4f21327dcb3dab36fbfdc2fcefb68975254631 Jul 15 17:42:27.344707 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:27.344663 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" event={"ID":"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b","Type":"ContainerStarted","Data":"7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef"} Jul 15 17:42:27.344707 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:27.344707 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" event={"ID":"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b","Type":"ContainerStarted","Data":"93345b0d9b62233050abb71cab4f21327dcb3dab36fbfdc2fcefb68975254631"} Jul 15 17:42:28.251466 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:28.251434 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99eb1e7f-0cbc-4e16-8b57-2481ae447f01" path="/var/lib/kubelet/pods/99eb1e7f-0cbc-4e16-8b57-2481ae447f01/volumes" Jul 15 17:42:37.061013 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:37.060928 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:37.061401 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:37.061190 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:37.073785 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:37.073763 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:37.094876 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:37.094816 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" podStartSLOduration=11.094802868 podStartE2EDuration="11.094802868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:42:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:42:27.373298889 +0000 UTC m=+2597.615294115" watchObservedRunningTime="2026-07-15 17:42:37.094802868 +0000 UTC m=+2607.336798119" Jul 15 17:42:37.412320 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:37.412293 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:42:46.742537 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:42:46.742500 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:43:05.752660 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:05.752604 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:43:12.261376 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:12.261338 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:43:12.362219 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:12.362188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:43:30.063303 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:30.063264 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2"] Jul 15 17:43:30.103371 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:30.103341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:43:30.857981 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:30.857936 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:43:45.083703 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.083635 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" podUID="32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" containerName="main" containerID="cri-o://7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef" gracePeriod=45 Jul 15 17:43:45.349917 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.349857 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:43:45.437463 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.437437 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tmp-dir\") pod \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " Jul 15 17:43:45.437636 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.437504 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-model-cache\") pod \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " Jul 15 17:43:45.437636 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.437537 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-dshm\") pod \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " Jul 15 17:43:45.437636 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.437562 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-home\") pod \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " Jul 15 17:43:45.437636 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.437625 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tls-certs\") pod \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " Jul 15 17:43:45.437877 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.437650 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcvrh\" (UniqueName: \"kubernetes.io/projected/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-kube-api-access-tcvrh\") pod \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\" (UID: \"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b\") " Jul 15 17:43:45.437877 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.437725 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" (UID: "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:43:45.437877 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.437807 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-model-cache" (OuterVolumeSpecName: "model-cache") pod "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" (UID: "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:43:45.438042 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.437934 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-home" (OuterVolumeSpecName: "home") pod "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" (UID: "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:43:45.438095 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.438079 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:43:45.438144 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.438100 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:43:45.438144 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.438114 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:43:45.439948 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.439927 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-dshm" (OuterVolumeSpecName: "dshm") pod "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" (UID: "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:43:45.440241 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.440224 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" (UID: "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:43:45.440291 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.440272 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-kube-api-access-tcvrh" (OuterVolumeSpecName: "kube-api-access-tcvrh") pod "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" (UID: "32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b"). InnerVolumeSpecName "kube-api-access-tcvrh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:43:45.538824 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.538797 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:43:45.538824 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.538821 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcvrh\" (UniqueName: \"kubernetes.io/projected/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-kube-api-access-tcvrh\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:43:45.538824 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.538829 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:43:45.768744 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.768646 2562 generic.go:356] "Generic (PLEG): container finished" podID="32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" containerID="7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef" exitCode=0 Jul 15 17:43:45.768744 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.768693 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" Jul 15 17:43:45.768744 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.768739 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" event={"ID":"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b","Type":"ContainerDied","Data":"7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef"} Jul 15 17:43:45.769060 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.768779 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2" event={"ID":"32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b","Type":"ContainerDied","Data":"93345b0d9b62233050abb71cab4f21327dcb3dab36fbfdc2fcefb68975254631"} Jul 15 17:43:45.769060 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.768796 2562 scope.go:117] "RemoveContainer" containerID="7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef" Jul 15 17:43:45.778272 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.778254 2562 scope.go:117] "RemoveContainer" containerID="7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef" Jul 15 17:43:45.778512 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:43:45.778495 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef\": container with ID starting with 7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef not found: ID does not exist" containerID="7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef" Jul 15 17:43:45.778559 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.778522 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef"} err="failed to get container status \"7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef\": rpc error: code = NotFound desc = could not find container \"7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef\": container with ID starting with 7872b5e1e6eaa102415dda57ce3a236e884fa73c5f3c20394b754c581b21c4ef not found: ID does not exist" Jul 15 17:43:45.789769 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.789747 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2"] Jul 15 17:43:45.792776 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:45.792755 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/sub-test-model-0bc296-kserve-68865c5b7-n2tz2"] Jul 15 17:43:46.251610 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:43:46.251567 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" path="/var/lib/kubelet/pods/32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b/volumes" Jul 15 17:44:05.596200 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:05.596116 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:44:06.824813 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:06.824783 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw"] Jul 15 17:44:06.825388 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:06.825371 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" containerName="main" Jul 15 17:44:06.825432 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:06.825393 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" containerName="main" Jul 15 17:44:06.825520 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:06.825509 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="32dc4ff3-62f3-401a-a8da-fe5fbd24dc0b" containerName="main" Jul 15 17:44:06.828990 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:06.828969 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:06.831863 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:06.831818 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0bc296\"/\"openshift-service-ca.crt\"" Jul 15 17:44:06.832909 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:06.832886 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0bc296\"/\"kube-root-ca.crt\"" Jul 15 17:44:06.832909 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:06.832903 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0bc296\"/\"rate-test-model-0bc296-kserve-self-signed-certs\"" Jul 15 17:44:06.833066 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:06.832893 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0bc296\"/\"default-dockercfg-tds97\"" Jul 15 17:44:06.838338 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:06.838316 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw"] Jul 15 17:44:06.854738 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:06.854708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:44:07.025252 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.025204 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-dshm\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.025252 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.025251 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-home\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.025497 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.025269 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-model-cache\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.025497 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.025308 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jhdq\" (UniqueName: \"kubernetes.io/projected/518037f2-ebff-42c9-89b5-e607b33041c6-kube-api-access-7jhdq\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.025497 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.025363 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-tmp-dir\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.025497 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.025450 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/518037f2-ebff-42c9-89b5-e607b33041c6-tls-certs\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.126162 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.126077 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-dshm\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.126162 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.126110 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-home\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.126162 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.126127 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-model-cache\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.126162 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.126142 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jhdq\" (UniqueName: \"kubernetes.io/projected/518037f2-ebff-42c9-89b5-e607b33041c6-kube-api-access-7jhdq\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.126162 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.126165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-tmp-dir\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.126518 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.126260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/518037f2-ebff-42c9-89b5-e607b33041c6-tls-certs\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.126584 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.126551 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-home\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.126644 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.126602 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-model-cache\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.126699 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.126647 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-tmp-dir\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.128540 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.128517 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-dshm\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.128716 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.128699 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/518037f2-ebff-42c9-89b5-e607b33041c6-tls-certs\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.134699 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.134680 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jhdq\" (UniqueName: \"kubernetes.io/projected/518037f2-ebff-42c9-89b5-e607b33041c6-kube-api-access-7jhdq\") pod \"rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.139523 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.139506 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:07.278796 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.278766 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw"] Jul 15 17:44:07.282098 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:44:07.282071 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod518037f2_ebff_42c9_89b5_e607b33041c6.slice/crio-6b82414f062acdfa45b2a193d10c7315a1a4e4400d7f48e3d1e10371c94d46de WatchSource:0}: Error finding container 6b82414f062acdfa45b2a193d10c7315a1a4e4400d7f48e3d1e10371c94d46de: Status 404 returned error can't find the container with id 6b82414f062acdfa45b2a193d10c7315a1a4e4400d7f48e3d1e10371c94d46de Jul 15 17:44:07.887847 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.887812 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" event={"ID":"518037f2-ebff-42c9-89b5-e607b33041c6","Type":"ContainerStarted","Data":"465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705"} Jul 15 17:44:07.888263 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.887862 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" event={"ID":"518037f2-ebff-42c9-89b5-e607b33041c6","Type":"ContainerStarted","Data":"6b82414f062acdfa45b2a193d10c7315a1a4e4400d7f48e3d1e10371c94d46de"} Jul 15 17:44:07.906033 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:07.905982 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" podStartSLOduration=1.905962575 podStartE2EDuration="1.905962575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:44:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:44:07.905805889 +0000 UTC m=+2698.147801116" watchObservedRunningTime="2026-07-15 17:44:07.905962575 +0000 UTC m=+2698.147957801" Jul 15 17:44:17.139753 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:17.139713 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:17.140147 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:17.139776 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:17.152170 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:17.152147 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:17.954577 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:17.954545 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:44:29.719518 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:29.719483 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:44:52.660161 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:52.660124 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:44:52.761022 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:44:52.760987 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:00.936492 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:00.936458 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:01.039211 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:01.039174 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:14.744549 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:14.744513 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw"] Jul 15 17:45:14.769113 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:14.769075 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:15.620226 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:15.620175 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:16.588968 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:16.588938 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7471-140678537932080"] Jul 15 17:45:16.592622 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:16.592605 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537932080" Jul 15 17:45:16.595146 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:16.595129 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9h7r7\"" Jul 15 17:45:16.599339 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:16.599318 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7471-140678537932080"] Jul 15 17:45:16.654731 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:16.654695 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6grbb\" (UniqueName: \"kubernetes.io/projected/bf89970e-e5e5-4430-817d-a59dc89b5956-kube-api-access-6grbb\") pod \"test-curl-7471-140678537932080\" (UID: \"bf89970e-e5e5-4430-817d-a59dc89b5956\") " pod="opendatahub/test-curl-7471-140678537932080" Jul 15 17:45:16.755406 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:16.755372 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6grbb\" (UniqueName: \"kubernetes.io/projected/bf89970e-e5e5-4430-817d-a59dc89b5956-kube-api-access-6grbb\") pod \"test-curl-7471-140678537932080\" (UID: \"bf89970e-e5e5-4430-817d-a59dc89b5956\") " pod="opendatahub/test-curl-7471-140678537932080" Jul 15 17:45:16.764162 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:16.764133 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6grbb\" (UniqueName: \"kubernetes.io/projected/bf89970e-e5e5-4430-817d-a59dc89b5956-kube-api-access-6grbb\") pod \"test-curl-7471-140678537932080\" (UID: \"bf89970e-e5e5-4430-817d-a59dc89b5956\") " pod="opendatahub/test-curl-7471-140678537932080" Jul 15 17:45:16.903124 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:16.903032 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537932080" Jul 15 17:45:17.046230 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:17.046203 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7471-140678537932080"] Jul 15 17:45:17.047500 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:45:17.047474 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf89970e_e5e5_4430_817d_a59dc89b5956.slice/crio-8434a1fc396fd3db3759c647596c7b14392b9396dd4de3851565a01b762af748 WatchSource:0}: Error finding container 8434a1fc396fd3db3759c647596c7b14392b9396dd4de3851565a01b762af748: Status 404 returned error can't find the container with id 8434a1fc396fd3db3759c647596c7b14392b9396dd4de3851565a01b762af748 Jul 15 17:45:17.048821 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:17.048803 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:45:17.280881 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:17.280788 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7471-140678537932080" event={"ID":"bf89970e-e5e5-4430-817d-a59dc89b5956","Type":"ContainerStarted","Data":"8434a1fc396fd3db3759c647596c7b14392b9396dd4de3851565a01b762af748"} Jul 15 17:45:18.287323 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:18.287288 2562 generic.go:356] "Generic (PLEG): container finished" podID="bf89970e-e5e5-4430-817d-a59dc89b5956" containerID="867e99422503ab697d89e808dee4acb1aa7d5af8521e7421127db2df4fd5923c" exitCode=0 Jul 15 17:45:18.287691 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:18.287372 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7471-140678537932080" event={"ID":"bf89970e-e5e5-4430-817d-a59dc89b5956","Type":"ContainerDied","Data":"867e99422503ab697d89e808dee4acb1aa7d5af8521e7421127db2df4fd5923c"} Jul 15 17:45:19.438781 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:19.438746 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537932080" Jul 15 17:45:19.461755 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:19.461727 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7471-140678537932080_bf89970e-e5e5-4430-817d-a59dc89b5956/test-curl-7471-140678537932080/0.log" Jul 15 17:45:19.490640 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:19.490612 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7471-140678537932080"] Jul 15 17:45:19.499142 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:19.499118 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7471-140678537932080"] Jul 15 17:45:19.581353 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:19.581329 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6grbb\" (UniqueName: \"kubernetes.io/projected/bf89970e-e5e5-4430-817d-a59dc89b5956-kube-api-access-6grbb\") pod \"bf89970e-e5e5-4430-817d-a59dc89b5956\" (UID: \"bf89970e-e5e5-4430-817d-a59dc89b5956\") " Jul 15 17:45:19.583563 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:19.583538 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf89970e-e5e5-4430-817d-a59dc89b5956-kube-api-access-6grbb" (OuterVolumeSpecName: "kube-api-access-6grbb") pod "bf89970e-e5e5-4430-817d-a59dc89b5956" (UID: "bf89970e-e5e5-4430-817d-a59dc89b5956"). InnerVolumeSpecName "kube-api-access-6grbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:45:19.682009 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:19.681973 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6grbb\" (UniqueName: \"kubernetes.io/projected/bf89970e-e5e5-4430-817d-a59dc89b5956-kube-api-access-6grbb\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:45:20.253298 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:20.253213 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf89970e-e5e5-4430-817d-a59dc89b5956" path="/var/lib/kubelet/pods/bf89970e-e5e5-4430-817d-a59dc89b5956/volumes" Jul 15 17:45:20.298676 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:20.298635 2562 scope.go:117] "RemoveContainer" containerID="867e99422503ab697d89e808dee4acb1aa7d5af8521e7421127db2df4fd5923c" Jul 15 17:45:20.298815 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:20.298660 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537932080" Jul 15 17:45:22.344589 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.344559 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7471-140678537949376"] Jul 15 17:45:22.345188 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.345170 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf89970e-e5e5-4430-817d-a59dc89b5956" containerName="test-curl-7471-140678537932080" Jul 15 17:45:22.345238 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.345192 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf89970e-e5e5-4430-817d-a59dc89b5956" containerName="test-curl-7471-140678537932080" Jul 15 17:45:22.345307 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.345296 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf89970e-e5e5-4430-817d-a59dc89b5956" containerName="test-curl-7471-140678537932080" Jul 15 17:45:22.350082 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.350027 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:22.352953 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.352935 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9h7r7\"" Jul 15 17:45:22.353136 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.353117 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7471-140678537949376"] Jul 15 17:45:22.406542 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.406517 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftsj9\" (UniqueName: \"kubernetes.io/projected/a44996ae-997b-4e98-b1c7-969cae534fd7-kube-api-access-ftsj9\") pod \"test-curl-7471-140678537949376\" (UID: \"a44996ae-997b-4e98-b1c7-969cae534fd7\") " pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:22.507113 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.507079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftsj9\" (UniqueName: \"kubernetes.io/projected/a44996ae-997b-4e98-b1c7-969cae534fd7-kube-api-access-ftsj9\") pod \"test-curl-7471-140678537949376\" (UID: \"a44996ae-997b-4e98-b1c7-969cae534fd7\") " pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:22.518653 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.518627 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftsj9\" (UniqueName: \"kubernetes.io/projected/a44996ae-997b-4e98-b1c7-969cae534fd7-kube-api-access-ftsj9\") pod \"test-curl-7471-140678537949376\" (UID: \"a44996ae-997b-4e98-b1c7-969cae534fd7\") " pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:22.661123 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.661044 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:22.797519 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:22.797498 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7471-140678537949376"] Jul 15 17:45:22.799020 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:45:22.798991 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda44996ae_997b_4e98_b1c7_969cae534fd7.slice/crio-6d68345800d0b6190f0dbd09b6e5984279002c961ab334aff1f8a8bee07f7eeb WatchSource:0}: Error finding container 6d68345800d0b6190f0dbd09b6e5984279002c961ab334aff1f8a8bee07f7eeb: Status 404 returned error can't find the container with id 6d68345800d0b6190f0dbd09b6e5984279002c961ab334aff1f8a8bee07f7eeb Jul 15 17:45:23.315434 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:23.315410 2562 generic.go:356] "Generic (PLEG): container finished" podID="a44996ae-997b-4e98-b1c7-969cae534fd7" containerID="4c5bbf54f0bfe8fea5c794a8918306a5c106096915d9a12bd3a43b7a362378b2" exitCode=0 Jul 15 17:45:23.315569 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:23.315492 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7471-140678537949376" event={"ID":"a44996ae-997b-4e98-b1c7-969cae534fd7","Type":"ContainerDied","Data":"4c5bbf54f0bfe8fea5c794a8918306a5c106096915d9a12bd3a43b7a362378b2"} Jul 15 17:45:23.315569 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:23.315526 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7471-140678537949376" event={"ID":"a44996ae-997b-4e98-b1c7-969cae534fd7","Type":"ContainerStarted","Data":"6d68345800d0b6190f0dbd09b6e5984279002c961ab334aff1f8a8bee07f7eeb"} Jul 15 17:45:24.458512 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.458488 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:24.478629 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.478595 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7471-140678537949376_a44996ae-997b-4e98-b1c7-969cae534fd7/test-curl-7471-140678537949376/0.log" Jul 15 17:45:24.502008 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.501969 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7471-140678537949376"] Jul 15 17:45:24.505981 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.505960 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7471-140678537949376"] Jul 15 17:45:24.525658 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.525634 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftsj9\" (UniqueName: \"kubernetes.io/projected/a44996ae-997b-4e98-b1c7-969cae534fd7-kube-api-access-ftsj9\") pod \"a44996ae-997b-4e98-b1c7-969cae534fd7\" (UID: \"a44996ae-997b-4e98-b1c7-969cae534fd7\") " Jul 15 17:45:24.534011 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.533977 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a44996ae-997b-4e98-b1c7-969cae534fd7-kube-api-access-ftsj9" (OuterVolumeSpecName: "kube-api-access-ftsj9") pod "a44996ae-997b-4e98-b1c7-969cae534fd7" (UID: "a44996ae-997b-4e98-b1c7-969cae534fd7"). InnerVolumeSpecName "kube-api-access-ftsj9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:45:24.626469 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.626434 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7471-140678537949376"] Jul 15 17:45:24.626851 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.626813 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftsj9\" (UniqueName: \"kubernetes.io/projected/a44996ae-997b-4e98-b1c7-969cae534fd7-kube-api-access-ftsj9\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:45:24.626991 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.626973 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a44996ae-997b-4e98-b1c7-969cae534fd7" containerName="test-curl-7471-140678537949376" Jul 15 17:45:24.627059 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.626995 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a44996ae-997b-4e98-b1c7-969cae534fd7" containerName="test-curl-7471-140678537949376" Jul 15 17:45:24.627119 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.627089 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a44996ae-997b-4e98-b1c7-969cae534fd7" containerName="test-curl-7471-140678537949376" Jul 15 17:45:24.630233 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.630214 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:24.636925 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.636900 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7471-140678537949376"] Jul 15 17:45:24.727948 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.727918 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlj62\" (UniqueName: \"kubernetes.io/projected/15db5fd7-ceb2-4628-a44a-63b6dfba3488-kube-api-access-xlj62\") pod \"test-curl-7471-140678537949376\" (UID: \"15db5fd7-ceb2-4628-a44a-63b6dfba3488\") " pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:24.829017 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.828980 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlj62\" (UniqueName: \"kubernetes.io/projected/15db5fd7-ceb2-4628-a44a-63b6dfba3488-kube-api-access-xlj62\") pod \"test-curl-7471-140678537949376\" (UID: \"15db5fd7-ceb2-4628-a44a-63b6dfba3488\") " pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:24.838483 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.838457 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlj62\" (UniqueName: \"kubernetes.io/projected/15db5fd7-ceb2-4628-a44a-63b6dfba3488-kube-api-access-xlj62\") pod \"test-curl-7471-140678537949376\" (UID: \"15db5fd7-ceb2-4628-a44a-63b6dfba3488\") " pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:24.942298 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:24.942220 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:25.083115 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:25.083082 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7471-140678537949376"] Jul 15 17:45:25.084488 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:45:25.084458 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15db5fd7_ceb2_4628_a44a_63b6dfba3488.slice/crio-4f1ff40df88cd37f63bd48c2a8959e354d0c5c0d308a1fd1a08aa1c44914f14f WatchSource:0}: Error finding container 4f1ff40df88cd37f63bd48c2a8959e354d0c5c0d308a1fd1a08aa1c44914f14f: Status 404 returned error can't find the container with id 4f1ff40df88cd37f63bd48c2a8959e354d0c5c0d308a1fd1a08aa1c44914f14f Jul 15 17:45:25.329277 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:25.329248 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d68345800d0b6190f0dbd09b6e5984279002c961ab334aff1f8a8bee07f7eeb" Jul 15 17:45:25.329277 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:25.329249 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:25.330767 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:25.330732 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7471-140678537949376" event={"ID":"15db5fd7-ceb2-4628-a44a-63b6dfba3488","Type":"ContainerStarted","Data":"4f1ff40df88cd37f63bd48c2a8959e354d0c5c0d308a1fd1a08aa1c44914f14f"} Jul 15 17:45:25.342611 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:25.342577 2562 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7471-140678537949376" oldPodUID="a44996ae-997b-4e98-b1c7-969cae534fd7" podUID="15db5fd7-ceb2-4628-a44a-63b6dfba3488" Jul 15 17:45:26.251792 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:26.251748 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a44996ae-997b-4e98-b1c7-969cae534fd7" path="/var/lib/kubelet/pods/a44996ae-997b-4e98-b1c7-969cae534fd7/volumes" Jul 15 17:45:26.338926 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:26.338890 2562 generic.go:356] "Generic (PLEG): container finished" podID="15db5fd7-ceb2-4628-a44a-63b6dfba3488" containerID="ee861acf2b893a2d0f98835e4302546d40c40a1d58149742ca67cf2ee3e6cee0" exitCode=0 Jul 15 17:45:26.339079 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:26.338988 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7471-140678537949376" event={"ID":"15db5fd7-ceb2-4628-a44a-63b6dfba3488","Type":"ContainerDied","Data":"ee861acf2b893a2d0f98835e4302546d40c40a1d58149742ca67cf2ee3e6cee0"} Jul 15 17:45:27.483773 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.483749 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:27.507669 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.507643 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7471-140678537949376_15db5fd7-ceb2-4628-a44a-63b6dfba3488/test-curl-7471-140678537949376/0.log" Jul 15 17:45:27.535775 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.535743 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7471-140678537949376"] Jul 15 17:45:27.539408 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.539384 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7471-140678537949376"] Jul 15 17:45:27.556690 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.556665 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlj62\" (UniqueName: \"kubernetes.io/projected/15db5fd7-ceb2-4628-a44a-63b6dfba3488-kube-api-access-xlj62\") pod \"15db5fd7-ceb2-4628-a44a-63b6dfba3488\" (UID: \"15db5fd7-ceb2-4628-a44a-63b6dfba3488\") " Jul 15 17:45:27.558896 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.558871 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15db5fd7-ceb2-4628-a44a-63b6dfba3488-kube-api-access-xlj62" (OuterVolumeSpecName: "kube-api-access-xlj62") pod "15db5fd7-ceb2-4628-a44a-63b6dfba3488" (UID: "15db5fd7-ceb2-4628-a44a-63b6dfba3488"). InnerVolumeSpecName "kube-api-access-xlj62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:45:27.658282 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.658208 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlj62\" (UniqueName: \"kubernetes.io/projected/15db5fd7-ceb2-4628-a44a-63b6dfba3488-kube-api-access-xlj62\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:45:27.662254 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.662228 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7471-140678537952400"] Jul 15 17:45:27.662660 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.662644 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15db5fd7-ceb2-4628-a44a-63b6dfba3488" containerName="test-curl-7471-140678537949376" Jul 15 17:45:27.662660 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.662661 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="15db5fd7-ceb2-4628-a44a-63b6dfba3488" containerName="test-curl-7471-140678537949376" Jul 15 17:45:27.662801 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.662750 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="15db5fd7-ceb2-4628-a44a-63b6dfba3488" containerName="test-curl-7471-140678537949376" Jul 15 17:45:27.665744 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.665727 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:27.673453 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.673431 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7471-140678537952400"] Jul 15 17:45:27.759537 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.759512 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw7k8\" (UniqueName: \"kubernetes.io/projected/febad1e8-87c5-4faf-a0c1-5357a1aecfe0-kube-api-access-qw7k8\") pod \"test-curl-7471-140678537952400\" (UID: \"febad1e8-87c5-4faf-a0c1-5357a1aecfe0\") " pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:27.860224 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.860195 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw7k8\" (UniqueName: \"kubernetes.io/projected/febad1e8-87c5-4faf-a0c1-5357a1aecfe0-kube-api-access-qw7k8\") pod \"test-curl-7471-140678537952400\" (UID: \"febad1e8-87c5-4faf-a0c1-5357a1aecfe0\") " pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:27.870061 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.870036 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw7k8\" (UniqueName: \"kubernetes.io/projected/febad1e8-87c5-4faf-a0c1-5357a1aecfe0-kube-api-access-qw7k8\") pod \"test-curl-7471-140678537952400\" (UID: \"febad1e8-87c5-4faf-a0c1-5357a1aecfe0\") " pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:27.976558 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:27.976485 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:28.113438 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:28.113409 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7471-140678537952400"] Jul 15 17:45:28.114706 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:45:28.114680 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfebad1e8_87c5_4faf_a0c1_5357a1aecfe0.slice/crio-fe4716f26573369cdfea3ee3fb833212765e7c4193583aced327b3a7d90ff668 WatchSource:0}: Error finding container fe4716f26573369cdfea3ee3fb833212765e7c4193583aced327b3a7d90ff668: Status 404 returned error can't find the container with id fe4716f26573369cdfea3ee3fb833212765e7c4193583aced327b3a7d90ff668 Jul 15 17:45:28.252569 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:28.252501 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15db5fd7-ceb2-4628-a44a-63b6dfba3488" path="/var/lib/kubelet/pods/15db5fd7-ceb2-4628-a44a-63b6dfba3488/volumes" Jul 15 17:45:28.351579 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:28.351552 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537949376" Jul 15 17:45:28.351735 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:28.351589 2562 scope.go:117] "RemoveContainer" containerID="ee861acf2b893a2d0f98835e4302546d40c40a1d58149742ca67cf2ee3e6cee0" Jul 15 17:45:28.353064 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:28.353034 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7471-140678537952400" event={"ID":"febad1e8-87c5-4faf-a0c1-5357a1aecfe0","Type":"ContainerStarted","Data":"fe4716f26573369cdfea3ee3fb833212765e7c4193583aced327b3a7d90ff668"} Jul 15 17:45:29.361112 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:29.361079 2562 generic.go:356] "Generic (PLEG): container finished" podID="febad1e8-87c5-4faf-a0c1-5357a1aecfe0" containerID="750a081000df064f02eeb5457e6831d1d75c74beaab7037132164d9db4d2fd90" exitCode=0 Jul 15 17:45:29.361521 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:29.361169 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7471-140678537952400" event={"ID":"febad1e8-87c5-4faf-a0c1-5357a1aecfe0","Type":"ContainerDied","Data":"750a081000df064f02eeb5457e6831d1d75c74beaab7037132164d9db4d2fd90"} Jul 15 17:45:29.772755 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:29.772659 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" podUID="518037f2-ebff-42c9-89b5-e607b33041c6" containerName="main" containerID="cri-o://465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705" gracePeriod=45 Jul 15 17:45:30.025697 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.025631 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:45:30.182324 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.182296 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-dshm\") pod \"518037f2-ebff-42c9-89b5-e607b33041c6\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " Jul 15 17:45:30.182324 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.182328 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-home\") pod \"518037f2-ebff-42c9-89b5-e607b33041c6\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " Jul 15 17:45:30.182519 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.182348 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jhdq\" (UniqueName: \"kubernetes.io/projected/518037f2-ebff-42c9-89b5-e607b33041c6-kube-api-access-7jhdq\") pod \"518037f2-ebff-42c9-89b5-e607b33041c6\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " Jul 15 17:45:30.182519 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.182378 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/518037f2-ebff-42c9-89b5-e607b33041c6-tls-certs\") pod \"518037f2-ebff-42c9-89b5-e607b33041c6\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " Jul 15 17:45:30.182519 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.182483 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-model-cache\") pod \"518037f2-ebff-42c9-89b5-e607b33041c6\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " Jul 15 17:45:30.182648 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.182551 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-tmp-dir\") pod \"518037f2-ebff-42c9-89b5-e607b33041c6\" (UID: \"518037f2-ebff-42c9-89b5-e607b33041c6\") " Jul 15 17:45:30.182648 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.182613 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-home" (OuterVolumeSpecName: "home") pod "518037f2-ebff-42c9-89b5-e607b33041c6" (UID: "518037f2-ebff-42c9-89b5-e607b33041c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:45:30.182821 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.182802 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-home\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:45:30.182978 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.182797 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-model-cache" (OuterVolumeSpecName: "model-cache") pod "518037f2-ebff-42c9-89b5-e607b33041c6" (UID: "518037f2-ebff-42c9-89b5-e607b33041c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:45:30.182978 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.182912 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "518037f2-ebff-42c9-89b5-e607b33041c6" (UID: "518037f2-ebff-42c9-89b5-e607b33041c6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:45:30.184679 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.184657 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-dshm" (OuterVolumeSpecName: "dshm") pod "518037f2-ebff-42c9-89b5-e607b33041c6" (UID: "518037f2-ebff-42c9-89b5-e607b33041c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:45:30.184679 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.184658 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/518037f2-ebff-42c9-89b5-e607b33041c6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "518037f2-ebff-42c9-89b5-e607b33041c6" (UID: "518037f2-ebff-42c9-89b5-e607b33041c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:45:30.184815 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.184719 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/518037f2-ebff-42c9-89b5-e607b33041c6-kube-api-access-7jhdq" (OuterVolumeSpecName: "kube-api-access-7jhdq") pod "518037f2-ebff-42c9-89b5-e607b33041c6" (UID: "518037f2-ebff-42c9-89b5-e607b33041c6"). InnerVolumeSpecName "kube-api-access-7jhdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:45:30.283703 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.283641 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-model-cache\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:45:30.283703 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.283663 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-tmp-dir\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:45:30.283703 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.283671 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/518037f2-ebff-42c9-89b5-e607b33041c6-dshm\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:45:30.283703 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.283679 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jhdq\" (UniqueName: \"kubernetes.io/projected/518037f2-ebff-42c9-89b5-e607b33041c6-kube-api-access-7jhdq\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:45:30.283703 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.283687 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/518037f2-ebff-42c9-89b5-e607b33041c6-tls-certs\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:45:30.368238 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.368203 2562 generic.go:356] "Generic (PLEG): container finished" podID="518037f2-ebff-42c9-89b5-e607b33041c6" containerID="465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705" exitCode=0 Jul 15 17:45:30.368670 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.368244 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" Jul 15 17:45:30.368670 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.368284 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" event={"ID":"518037f2-ebff-42c9-89b5-e607b33041c6","Type":"ContainerDied","Data":"465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705"} Jul 15 17:45:30.368670 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.368325 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw" event={"ID":"518037f2-ebff-42c9-89b5-e607b33041c6","Type":"ContainerDied","Data":"6b82414f062acdfa45b2a193d10c7315a1a4e4400d7f48e3d1e10371c94d46de"} Jul 15 17:45:30.368670 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.368343 2562 scope.go:117] "RemoveContainer" containerID="465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705" Jul 15 17:45:30.377739 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.377712 2562 scope.go:117] "RemoveContainer" containerID="465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705" Jul 15 17:45:30.378031 ip-10-0-135-132 kubenswrapper[2562]: E0715 17:45:30.378010 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705\": container with ID starting with 465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705 not found: ID does not exist" containerID="465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705" Jul 15 17:45:30.378102 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.378043 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705"} err="failed to get container status \"465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705\": rpc error: code = NotFound desc = could not find container \"465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705\": container with ID starting with 465f31c560019af0030987f189ee0267a0a75daadb94fc9fb79fcde71dc4e705 not found: ID does not exist" Jul 15 17:45:30.387118 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.387094 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw"] Jul 15 17:45:30.390910 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.390885 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0bc296/rate-test-model-0bc296-kserve-68b9d67cc5-f8mdw"] Jul 15 17:45:30.490040 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.490014 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:30.510564 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.510539 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7471-140678537952400_febad1e8-87c5-4faf-a0c1-5357a1aecfe0/test-curl-7471-140678537952400/0.log" Jul 15 17:45:30.535279 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.535186 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7471-140678537952400"] Jul 15 17:45:30.538937 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.538903 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7471-140678537952400"] Jul 15 17:45:30.585743 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.585710 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw7k8\" (UniqueName: \"kubernetes.io/projected/febad1e8-87c5-4faf-a0c1-5357a1aecfe0-kube-api-access-qw7k8\") pod \"febad1e8-87c5-4faf-a0c1-5357a1aecfe0\" (UID: \"febad1e8-87c5-4faf-a0c1-5357a1aecfe0\") " Jul 15 17:45:30.587829 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.587810 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/febad1e8-87c5-4faf-a0c1-5357a1aecfe0-kube-api-access-qw7k8" (OuterVolumeSpecName: "kube-api-access-qw7k8") pod "febad1e8-87c5-4faf-a0c1-5357a1aecfe0" (UID: "febad1e8-87c5-4faf-a0c1-5357a1aecfe0"). InnerVolumeSpecName "kube-api-access-qw7k8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:45:30.658326 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.658295 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7471-140678537952400"] Jul 15 17:45:30.658927 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.658911 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="518037f2-ebff-42c9-89b5-e607b33041c6" containerName="main" Jul 15 17:45:30.658991 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.658931 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="518037f2-ebff-42c9-89b5-e607b33041c6" containerName="main" Jul 15 17:45:30.658991 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.658953 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="febad1e8-87c5-4faf-a0c1-5357a1aecfe0" containerName="test-curl-7471-140678537952400" Jul 15 17:45:30.658991 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.658959 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="febad1e8-87c5-4faf-a0c1-5357a1aecfe0" containerName="test-curl-7471-140678537952400" Jul 15 17:45:30.659088 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.659031 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="febad1e8-87c5-4faf-a0c1-5357a1aecfe0" containerName="test-curl-7471-140678537952400" Jul 15 17:45:30.659088 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.659042 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="518037f2-ebff-42c9-89b5-e607b33041c6" containerName="main" Jul 15 17:45:30.662102 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.662083 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:30.669852 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.669818 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7471-140678537952400"] Jul 15 17:45:30.686739 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.686708 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw7k8\" (UniqueName: \"kubernetes.io/projected/febad1e8-87c5-4faf-a0c1-5357a1aecfe0-kube-api-access-qw7k8\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:45:30.787536 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.787509 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhz45\" (UniqueName: \"kubernetes.io/projected/ef866188-7083-45f2-9ac3-98a2bfcaf9c2-kube-api-access-hhz45\") pod \"test-curl-7471-140678537952400\" (UID: \"ef866188-7083-45f2-9ac3-98a2bfcaf9c2\") " pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:30.888446 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.888419 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhz45\" (UniqueName: \"kubernetes.io/projected/ef866188-7083-45f2-9ac3-98a2bfcaf9c2-kube-api-access-hhz45\") pod \"test-curl-7471-140678537952400\" (UID: \"ef866188-7083-45f2-9ac3-98a2bfcaf9c2\") " pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:30.897168 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.897144 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhz45\" (UniqueName: \"kubernetes.io/projected/ef866188-7083-45f2-9ac3-98a2bfcaf9c2-kube-api-access-hhz45\") pod \"test-curl-7471-140678537952400\" (UID: \"ef866188-7083-45f2-9ac3-98a2bfcaf9c2\") " pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:30.972869 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:30.972822 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:31.128435 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:31.128410 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7471-140678537952400"] Jul 15 17:45:31.130282 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:45:31.130260 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef866188_7083_45f2_9ac3_98a2bfcaf9c2.slice/crio-77503021133ef4cfe3b6c61dfd0beb04126dc66ecf367398d569e53d5aae5dcc WatchSource:0}: Error finding container 77503021133ef4cfe3b6c61dfd0beb04126dc66ecf367398d569e53d5aae5dcc: Status 404 returned error can't find the container with id 77503021133ef4cfe3b6c61dfd0beb04126dc66ecf367398d569e53d5aae5dcc Jul 15 17:45:31.374902 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:31.374800 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe4716f26573369cdfea3ee3fb833212765e7c4193583aced327b3a7d90ff668" Jul 15 17:45:31.374902 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:31.374802 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:31.376107 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:31.376077 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7471-140678537952400" event={"ID":"ef866188-7083-45f2-9ac3-98a2bfcaf9c2","Type":"ContainerStarted","Data":"77503021133ef4cfe3b6c61dfd0beb04126dc66ecf367398d569e53d5aae5dcc"} Jul 15 17:45:31.387414 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:31.387391 2562 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7471-140678537952400" oldPodUID="febad1e8-87c5-4faf-a0c1-5357a1aecfe0" podUID="ef866188-7083-45f2-9ac3-98a2bfcaf9c2" Jul 15 17:45:32.251673 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:32.251638 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="518037f2-ebff-42c9-89b5-e607b33041c6" path="/var/lib/kubelet/pods/518037f2-ebff-42c9-89b5-e607b33041c6/volumes" Jul 15 17:45:32.252015 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:32.252001 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="febad1e8-87c5-4faf-a0c1-5357a1aecfe0" path="/var/lib/kubelet/pods/febad1e8-87c5-4faf-a0c1-5357a1aecfe0/volumes" Jul 15 17:45:32.384137 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:32.384108 2562 generic.go:356] "Generic (PLEG): container finished" podID="ef866188-7083-45f2-9ac3-98a2bfcaf9c2" containerID="bd93a92a032a39445efb9ac551394e8216632df023c6db5b9aba2d78f1ff4492" exitCode=0 Jul 15 17:45:32.384496 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:32.384161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7471-140678537952400" event={"ID":"ef866188-7083-45f2-9ac3-98a2bfcaf9c2","Type":"ContainerDied","Data":"bd93a92a032a39445efb9ac551394e8216632df023c6db5b9aba2d78f1ff4492"} Jul 15 17:45:33.530280 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:33.530257 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:33.556854 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:33.556817 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7471-140678537952400_ef866188-7083-45f2-9ac3-98a2bfcaf9c2/test-curl-7471-140678537952400/0.log" Jul 15 17:45:33.589373 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:33.589344 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7471-140678537952400"] Jul 15 17:45:33.596933 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:33.596333 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7471-140678537952400"] Jul 15 17:45:33.615989 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:33.615956 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhz45\" (UniqueName: \"kubernetes.io/projected/ef866188-7083-45f2-9ac3-98a2bfcaf9c2-kube-api-access-hhz45\") pod \"ef866188-7083-45f2-9ac3-98a2bfcaf9c2\" (UID: \"ef866188-7083-45f2-9ac3-98a2bfcaf9c2\") " Jul 15 17:45:33.618279 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:33.618253 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef866188-7083-45f2-9ac3-98a2bfcaf9c2-kube-api-access-hhz45" (OuterVolumeSpecName: "kube-api-access-hhz45") pod "ef866188-7083-45f2-9ac3-98a2bfcaf9c2" (UID: "ef866188-7083-45f2-9ac3-98a2bfcaf9c2"). InnerVolumeSpecName "kube-api-access-hhz45". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:45:33.717206 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:33.717142 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhz45\" (UniqueName: \"kubernetes.io/projected/ef866188-7083-45f2-9ac3-98a2bfcaf9c2-kube-api-access-hhz45\") on node \"ip-10-0-135-132.ec2.internal\" DevicePath \"\"" Jul 15 17:45:34.251682 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:34.251650 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef866188-7083-45f2-9ac3-98a2bfcaf9c2" path="/var/lib/kubelet/pods/ef866188-7083-45f2-9ac3-98a2bfcaf9c2/volumes" Jul 15 17:45:34.377737 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:34.377687 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:34.400877 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:34.400854 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7471-140678537952400" Jul 15 17:45:34.401057 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:34.400863 2562 scope.go:117] "RemoveContainer" containerID="bd93a92a032a39445efb9ac551394e8216632df023c6db5b9aba2d78f1ff4492" Jul 15 17:45:34.479110 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:34.479070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:44.318858 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:44.318802 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:48.946543 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:48.946504 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:55.113429 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:55.113389 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:55.181849 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:55.181798 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:59.688167 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:59.688125 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:45:59.957216 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:45:59.957133 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:00.714005 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:00.713973 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:00.814559 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:00.814515 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:10.647868 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:10.647817 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:15.195342 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:15.195303 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:21.019275 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:21.019240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:21.118813 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:21.118775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:25.954710 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:25.954671 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:26.198005 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:26.197968 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:29.600688 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:29.600658 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:32.586225 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:32.586180 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:32.642346 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:32.642317 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:32.656619 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:32.656596 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6964775bf7-2npjj_fcce1a78-bd24-4782-9f24-d14c5839b875/manager/0.log" Jul 15 17:46:33.155194 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:33.155153 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fdf764586-rsdsj_53c2b68b-d7de-480a-b5df-38e853644702/copy-manifests/0.log" Jul 15 17:46:33.174514 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:33.174490 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fdf764586-rsdsj_53c2b68b-d7de-480a-b5df-38e853644702/manager/0.log" Jul 15 17:46:33.688294 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:33.688257 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595df7b4f4-26wcc_69ffda4d-02af-4f53-969a-e98bcec2ae58/maas-api/0.log" Jul 15 17:46:36.326574 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.326542 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4_cb41f6d0-3ec7-4d42-956e-c7a8819d3643/extract/0.log" Jul 15 17:46:36.337219 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.337172 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4_cb41f6d0-3ec7-4d42-956e-c7a8819d3643/util/0.log" Jul 15 17:46:36.344358 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.344324 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4_cb41f6d0-3ec7-4d42-956e-c7a8819d3643/pull/0.log" Jul 15 17:46:36.465332 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.465299 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4_b83c0568-810f-42bb-88c2-4fc3551f803b/util/0.log" Jul 15 17:46:36.471199 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.471173 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4_b83c0568-810f-42bb-88c2-4fc3551f803b/pull/0.log" Jul 15 17:46:36.476947 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.476915 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4_b83c0568-810f-42bb-88c2-4fc3551f803b/extract/0.log" Jul 15 17:46:36.598463 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.598375 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl_31d68995-a464-4438-9bd6-a813f42c9156/extract/0.log" Jul 15 17:46:36.604205 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.604183 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl_31d68995-a464-4438-9bd6-a813f42c9156/util/0.log" Jul 15 17:46:36.610045 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.610024 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl_31d68995-a464-4438-9bd6-a813f42c9156/pull/0.log" Jul 15 17:46:36.727163 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.727138 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp_b89bf294-eb84-46b3-8cbc-8c96cfae52c7/util/0.log" Jul 15 17:46:36.733739 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.733716 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp_b89bf294-eb84-46b3-8cbc-8c96cfae52c7/pull/0.log" Jul 15 17:46:36.740333 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:36.740317 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp_b89bf294-eb84-46b3-8cbc-8c96cfae52c7/extract/0.log" Jul 15 17:46:37.003088 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:37.002996 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vjz6k_c06f6746-1f15-43bb-811f-84bcb0e5afdf/manager/0.log" Jul 15 17:46:37.228032 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:37.228008 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-q9ncs_f72d49b6-73af-48a0-8a20-c84388cfd160/kuadrant-console-plugin/0.log" Jul 15 17:46:37.360760 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:37.360719 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-84mk7_7b640a40-926d-4814-9982-23906b95e15c/registry-server/0.log" Jul 15 17:46:37.484404 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:37.484373 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zgnrn_6aecc0cc-e91a-4acf-86af-a88f46082860/manager/0.log" Jul 15 17:46:37.717936 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:37.717865 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2rqz4_18063618-b182-4e55-a8ac-2d08c7a8f0a4/manager/0.log" Jul 15 17:46:38.700160 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:38.700129 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jx5tt_fbff51cd-c23a-4650-ab5a-e9b80283d1de/payload-pre-processing/0.log" Jul 15 17:46:38.834886 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:38.834854 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-vqxgw_0a81cd00-394e-4fd4-8368-1f2ba0be9e57/payload-processing/0.log" Jul 15 17:46:39.432764 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:39.432737 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-wphhp_25d363eb-fc26-45cc-bca3-8788a082ab4a/main/0.log" Jul 15 17:46:39.660332 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:39.660300 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9z2x6_160677ed-43ed-4ee3-b1fc-0688c32a0f37/main/0.log" Jul 15 17:46:39.908598 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:39.908571 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-7nbbq_f91406d7-1c97-4c45-97dc-f21e6af07f19/main/0.log" Jul 15 17:46:45.545258 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:45.545222 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:45.642227 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:45.642192 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" limit=1024 Jul 15 17:46:48.498401 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:48.498372 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6r8xd_f4c52b79-c004-44fb-84b8-ee8c8a9ca5ca/global-pull-secret-syncer/0.log" Jul 15 17:46:48.669267 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:48.669236 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-87dcf_f19cf2f1-fb21-4d7f-add5-939f1dec2fca/konnectivity-agent/0.log" Jul 15 17:46:48.819634 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:48.819563 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-132.ec2.internal_b41c58adddba08521de05a44de7576c8/haproxy/0.log" Jul 15 17:46:53.506785 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.506749 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4_cb41f6d0-3ec7-4d42-956e-c7a8819d3643/extract/0.log" Jul 15 17:46:53.530224 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.530199 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4_cb41f6d0-3ec7-4d42-956e-c7a8819d3643/util/0.log" Jul 15 17:46:53.560604 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.560579 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xv2m4_cb41f6d0-3ec7-4d42-956e-c7a8819d3643/pull/0.log" Jul 15 17:46:53.590698 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.590673 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4_b83c0568-810f-42bb-88c2-4fc3551f803b/extract/0.log" Jul 15 17:46:53.614644 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.614625 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4_b83c0568-810f-42bb-88c2-4fc3551f803b/util/0.log" Jul 15 17:46:53.639122 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.639101 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0njnf4_b83c0568-810f-42bb-88c2-4fc3551f803b/pull/0.log" Jul 15 17:46:53.673209 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.673186 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl_31d68995-a464-4438-9bd6-a813f42c9156/extract/0.log" Jul 15 17:46:53.698497 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.698479 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl_31d68995-a464-4438-9bd6-a813f42c9156/util/0.log" Jul 15 17:46:53.732184 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.732163 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jxbl_31d68995-a464-4438-9bd6-a813f42c9156/pull/0.log" Jul 15 17:46:53.780875 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.780796 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp_b89bf294-eb84-46b3-8cbc-8c96cfae52c7/extract/0.log" Jul 15 17:46:53.813982 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.813959 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp_b89bf294-eb84-46b3-8cbc-8c96cfae52c7/util/0.log" Jul 15 17:46:53.841076 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:53.841054 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14n2gp_b89bf294-eb84-46b3-8cbc-8c96cfae52c7/pull/0.log" Jul 15 17:46:54.334668 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:54.334635 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vjz6k_c06f6746-1f15-43bb-811f-84bcb0e5afdf/manager/0.log" Jul 15 17:46:54.427705 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:54.427673 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-q9ncs_f72d49b6-73af-48a0-8a20-c84388cfd160/kuadrant-console-plugin/0.log" Jul 15 17:46:54.481700 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:54.481671 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-84mk7_7b640a40-926d-4814-9982-23906b95e15c/registry-server/0.log" Jul 15 17:46:54.663677 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:54.663597 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zgnrn_6aecc0cc-e91a-4acf-86af-a88f46082860/manager/0.log" Jul 15 17:46:54.786258 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:54.786222 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2rqz4_18063618-b182-4e55-a8ac-2d08c7a8f0a4/manager/0.log" Jul 15 17:46:56.332577 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.332549 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_87a18837-b8d6-47ce-89d7-bc3fbfc1cc36/alertmanager/0.log" Jul 15 17:46:56.353913 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.353887 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_87a18837-b8d6-47ce-89d7-bc3fbfc1cc36/config-reloader/0.log" Jul 15 17:46:56.375459 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.375432 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_87a18837-b8d6-47ce-89d7-bc3fbfc1cc36/kube-rbac-proxy-web/0.log" Jul 15 17:46:56.395244 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.395220 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_87a18837-b8d6-47ce-89d7-bc3fbfc1cc36/kube-rbac-proxy/0.log" Jul 15 17:46:56.417060 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.417032 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_87a18837-b8d6-47ce-89d7-bc3fbfc1cc36/kube-rbac-proxy-metric/0.log" Jul 15 17:46:56.437533 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.437517 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_87a18837-b8d6-47ce-89d7-bc3fbfc1cc36/prom-label-proxy/0.log" Jul 15 17:46:56.458179 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.458156 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_87a18837-b8d6-47ce-89d7-bc3fbfc1cc36/init-config-reloader/0.log" Jul 15 17:46:56.526453 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.526425 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-lpq5l_8506444a-b35c-474f-84e0-e3bceb00271a/kube-state-metrics/0.log" Jul 15 17:46:56.543991 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.543967 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-lpq5l_8506444a-b35c-474f-84e0-e3bceb00271a/kube-rbac-proxy-main/0.log" Jul 15 17:46:56.567867 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.567821 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-lpq5l_8506444a-b35c-474f-84e0-e3bceb00271a/kube-rbac-proxy-self/0.log" Jul 15 17:46:56.621713 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.621635 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-rctbg_9a7be7b5-4e96-4075-9880-ed9aed255eab/monitoring-plugin/0.log" Jul 15 17:46:56.800907 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.800858 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vvmkg_7e14736e-6d94-4699-89e4-3a05998d1afe/node-exporter/0.log" Jul 15 17:46:56.819620 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.819598 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vvmkg_7e14736e-6d94-4699-89e4-3a05998d1afe/kube-rbac-proxy/0.log" Jul 15 17:46:56.842445 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.842421 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vvmkg_7e14736e-6d94-4699-89e4-3a05998d1afe/init-textfile/0.log" Jul 15 17:46:56.872857 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.872761 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-npdxr_7adfd351-5cdc-43c3-8684-553480b8ab6f/kube-rbac-proxy-main/0.log" Jul 15 17:46:56.893143 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.893120 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-npdxr_7adfd351-5cdc-43c3-8684-553480b8ab6f/kube-rbac-proxy-self/0.log" Jul 15 17:46:56.914405 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:56.914377 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-npdxr_7adfd351-5cdc-43c3-8684-553480b8ab6f/openshift-state-metrics/0.log" Jul 15 17:46:57.207548 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:57.207462 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c4b99847c-2vf5v_7ffce7c9-7583-4e7a-a194-8b0827bc2ee1/telemeter-client/0.log" Jul 15 17:46:57.229961 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:57.229937 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c4b99847c-2vf5v_7ffce7c9-7583-4e7a-a194-8b0827bc2ee1/reload/0.log" Jul 15 17:46:57.249893 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:57.249867 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c4b99847c-2vf5v_7ffce7c9-7583-4e7a-a194-8b0827bc2ee1/kube-rbac-proxy/0.log" Jul 15 17:46:57.278510 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:57.278484 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749db7cfc4-tw8m9_1553e479-bbd5-40d4-b7b1-060572599bb9/thanos-query/0.log" Jul 15 17:46:57.309853 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:57.309809 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749db7cfc4-tw8m9_1553e479-bbd5-40d4-b7b1-060572599bb9/kube-rbac-proxy-web/0.log" Jul 15 17:46:57.329691 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:57.329666 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749db7cfc4-tw8m9_1553e479-bbd5-40d4-b7b1-060572599bb9/kube-rbac-proxy/0.log" Jul 15 17:46:57.348399 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:57.348362 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749db7cfc4-tw8m9_1553e479-bbd5-40d4-b7b1-060572599bb9/prom-label-proxy/0.log" Jul 15 17:46:57.367745 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:57.367725 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749db7cfc4-tw8m9_1553e479-bbd5-40d4-b7b1-060572599bb9/kube-rbac-proxy-rules/0.log" Jul 15 17:46:57.385842 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:57.385820 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749db7cfc4-tw8m9_1553e479-bbd5-40d4-b7b1-060572599bb9/kube-rbac-proxy-metrics/0.log" Jul 15 17:46:58.809305 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.809268 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh"] Jul 15 17:46:58.809867 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.809824 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef866188-7083-45f2-9ac3-98a2bfcaf9c2" containerName="test-curl-7471-140678537952400" Jul 15 17:46:58.809867 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.809861 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef866188-7083-45f2-9ac3-98a2bfcaf9c2" containerName="test-curl-7471-140678537952400" Jul 15 17:46:58.810029 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.809989 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef866188-7083-45f2-9ac3-98a2bfcaf9c2" containerName="test-curl-7471-140678537952400" Jul 15 17:46:58.813245 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.813225 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:58.816142 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.816123 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kwr2v\"/\"default-dockercfg-kt2q7\"" Jul 15 17:46:58.816404 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.816381 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwr2v\"/\"openshift-service-ca.crt\"" Jul 15 17:46:58.817114 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.817097 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwr2v\"/\"kube-root-ca.crt\"" Jul 15 17:46:58.821801 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.821775 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh"] Jul 15 17:46:58.977309 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.977274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-lib-modules\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:58.977455 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.977344 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2whbr\" (UniqueName: \"kubernetes.io/projected/a45053df-e7fe-4bb1-a46d-f229845ed871-kube-api-access-2whbr\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:58.977455 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.977387 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-sys\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:58.977455 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.977435 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-proc\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:58.977587 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:58.977505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-podres\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.078761 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.078732 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-podres\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.078944 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.078777 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-lib-modules\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.078944 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.078796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2whbr\" (UniqueName: \"kubernetes.io/projected/a45053df-e7fe-4bb1-a46d-f229845ed871-kube-api-access-2whbr\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.078944 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.078820 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-sys\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.078944 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.078879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-proc\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.078944 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.078936 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-podres\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.079211 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.078956 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-sys\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.079211 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.078936 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-lib-modules\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.079211 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.078989 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a45053df-e7fe-4bb1-a46d-f229845ed871-proc\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.088356 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.088335 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2whbr\" (UniqueName: \"kubernetes.io/projected/a45053df-e7fe-4bb1-a46d-f229845ed871-kube-api-access-2whbr\") pod \"perf-node-gather-daemonset-7fldh\" (UID: \"a45053df-e7fe-4bb1-a46d-f229845ed871\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.125404 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.125387 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.262667 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.262643 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh"] Jul 15 17:46:59.263886 ip-10-0-135-132 kubenswrapper[2562]: W0715 17:46:59.263861 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda45053df_e7fe_4bb1_a46d_f229845ed871.slice/crio-e6bc2f94995b707497f4f49cff8b738a75a650dcd23f8911a69babe7728ed4ed WatchSource:0}: Error finding container e6bc2f94995b707497f4f49cff8b738a75a650dcd23f8911a69babe7728ed4ed: Status 404 returned error can't find the container with id e6bc2f94995b707497f4f49cff8b738a75a650dcd23f8911a69babe7728ed4ed Jul 15 17:46:59.882280 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.882247 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" event={"ID":"a45053df-e7fe-4bb1-a46d-f229845ed871","Type":"ContainerStarted","Data":"dc107c38e47c01bf6acf5a275530c75d6b3b22522c49154ca093f469af41bdad"} Jul 15 17:46:59.882631 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.882399 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:46:59.882631 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.882434 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" event={"ID":"a45053df-e7fe-4bb1-a46d-f229845ed871","Type":"ContainerStarted","Data":"e6bc2f94995b707497f4f49cff8b738a75a650dcd23f8911a69babe7728ed4ed"} Jul 15 17:46:59.901909 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:46:59.901856 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" podStartSLOduration=1.901827951 podStartE2EDuration="1.901827951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 17:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:46:59.899575417 +0000 UTC m=+2870.141570643" watchObservedRunningTime="2026-07-15 17:46:59.901827951 +0000 UTC m=+2870.143823194" Jul 15 17:47:01.465271 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:01.465194 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-td2hl_c86c79e5-143a-4be2-b192-8192c6dedc20/dns/0.log" Jul 15 17:47:01.485863 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:01.485817 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-td2hl_c86c79e5-143a-4be2-b192-8192c6dedc20/kube-rbac-proxy/0.log" Jul 15 17:47:01.538236 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:01.538207 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sqd2w_2a700f8f-0ac0-4af0-a190-c8865d5db3e9/dns-node-resolver/0.log" Jul 15 17:47:02.140463 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:02.140426 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l46zz_435df3eb-d81b-471e-b851-dbb91a35dbce/node-ca/0.log" Jul 15 17:47:03.324474 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:03.324429 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jx5tt_fbff51cd-c23a-4650-ab5a-e9b80283d1de/payload-pre-processing/0.log" Jul 15 17:47:03.363194 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:03.363143 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-vqxgw_0a81cd00-394e-4fd4-8368-1f2ba0be9e57/payload-processing/0.log" Jul 15 17:47:04.113993 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:04.113968 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rjhcq_6283c946-6d33-4e68-9bd5-94ff19b00c31/serve-healthcheck-canary/0.log" Jul 15 17:47:04.729530 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:04.729501 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bbtx8_5ba4bd8c-b016-4515-865b-6977c37af13e/kube-rbac-proxy/0.log" Jul 15 17:47:04.746677 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:04.746657 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bbtx8_5ba4bd8c-b016-4515-865b-6977c37af13e/exporter/0.log" Jul 15 17:47:04.767820 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:04.767802 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bbtx8_5ba4bd8c-b016-4515-865b-6977c37af13e/extractor/0.log" Jul 15 17:47:05.898091 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:05.898065 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-7fldh" Jul 15 17:47:07.331053 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:07.331011 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6964775bf7-2npjj_fcce1a78-bd24-4782-9f24-d14c5839b875/manager/0.log" Jul 15 17:47:07.589987 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:07.589898 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fdf764586-rsdsj_53c2b68b-d7de-480a-b5df-38e853644702/manager/0.log" Jul 15 17:47:07.607318 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:07.607292 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fdf764586-rsdsj_53c2b68b-d7de-480a-b5df-38e853644702/copy-manifests/0.log" Jul 15 17:47:09.048146 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:09.048113 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6744c79bb-m9dfv_38bc0cec-2380-4b99-a0e1-71cb3e32ad37/manager/0.log" Jul 15 17:47:09.095702 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:09.095674 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rbmns_b8a8b2c7-60fb-4557-9814-fb2bb412b2e5/openshift-lws-operator/0.log" Jul 15 17:47:09.270019 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:09.269931 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jx5tt_fbff51cd-c23a-4650-ab5a-e9b80283d1de/payload-pre-processing/0.log" Jul 15 17:47:09.299054 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:09.299023 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-vqxgw_0a81cd00-394e-4fd4-8368-1f2ba0be9e57/payload-processing/0.log" Jul 15 17:47:10.811600 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:10.811488 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jx5tt_fbff51cd-c23a-4650-ab5a-e9b80283d1de/payload-pre-processing/0.log" Jul 15 17:47:10.842616 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:10.842558 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-vqxgw_0a81cd00-394e-4fd4-8368-1f2ba0be9e57/payload-processing/0.log" Jul 15 17:47:15.529770 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:15.529735 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cxs9l_35c01559-3824-4e5c-bfea-71ead02ea8ed/kube-multus-additional-cni-plugins/0.log" Jul 15 17:47:15.552728 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:15.552700 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cxs9l_35c01559-3824-4e5c-bfea-71ead02ea8ed/egress-router-binary-copy/0.log" Jul 15 17:47:15.576237 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:15.576216 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cxs9l_35c01559-3824-4e5c-bfea-71ead02ea8ed/cni-plugins/0.log" Jul 15 17:47:15.599636 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:15.599611 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cxs9l_35c01559-3824-4e5c-bfea-71ead02ea8ed/bond-cni-plugin/0.log" Jul 15 17:47:15.621430 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:15.621413 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cxs9l_35c01559-3824-4e5c-bfea-71ead02ea8ed/routeoverride-cni/0.log" Jul 15 17:47:15.647250 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:15.647231 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cxs9l_35c01559-3824-4e5c-bfea-71ead02ea8ed/whereabouts-cni-bincopy/0.log" Jul 15 17:47:15.668157 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:15.668143 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cxs9l_35c01559-3824-4e5c-bfea-71ead02ea8ed/whereabouts-cni/0.log" Jul 15 17:47:15.906750 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:15.906718 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-w5h4s_652c50d6-b43e-4f5c-8cc2-70d2155a9188/kube-multus/0.log" Jul 15 17:47:15.931281 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:15.931258 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qwxnw_8dd93cd1-bc16-44a6-83fb-a0861aa19e26/network-metrics-daemon/0.log" Jul 15 17:47:15.947760 ip-10-0-135-132 kubenswrapper[2562]: I0715 17:47:15.947734 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qwxnw_8dd93cd1-bc16-44a6-83fb-a0861aa19e26/kube-rbac-proxy/0.log"