Jul 06 19:22:58.219977 ip-10-0-131-35 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 19:22:58.219991 ip-10-0-131-35 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 19:22:58.220001 ip-10-0-131-35 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 19:22:58.220337 ip-10-0-131-35 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 19:23:08.386271 ip-10-0-131-35 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 19:23:08.386290 ip-10-0-131-35 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 715fde44ad0448688adf7bd98b407b04 -- Jul 06 19:24:49.370229 ip-10-0-131-35 systemd[1]: Starting Kubernetes Kubelet... Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.800846 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806868 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806886 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806890 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806893 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806896 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806899 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806917 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:24:49.892845 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806921 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:24:49.877308 ip-10-0-131-35 systemd[1]: Started Kubernetes Kubelet. Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806924 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806927 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806929 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806932 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806935 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806947 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806950 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806953 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806956 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806959 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806962 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806965 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806968 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806970 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806973 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806976 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806980 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806982 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806985 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:24:49.895586 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806988 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806990 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806993 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806996 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.806998 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807001 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807004 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807007 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807009 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807013 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807015 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807018 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807021 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807023 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807026 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807031 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807034 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807037 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807039 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807044 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:24:49.897923 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807046 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807049 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807052 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807055 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807059 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807061 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807064 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807067 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807070 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807072 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807075 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807080 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807083 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807086 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807089 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807092 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807095 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807097 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807100 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807103 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:24:49.900703 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807105 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807108 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807111 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807113 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807116 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807119 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807122 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807124 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807128 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807130 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807135 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807138 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807141 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807144 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807147 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807150 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807153 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807156 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807160 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:24:49.915578 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807164 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807167 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807169 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807172 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807176 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807179 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807182 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807185 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807187 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807190 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807193 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807197 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.807200 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808000 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808010 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808013 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808016 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808019 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808021 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:24:49.917434 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808024 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808027 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808030 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808034 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808036 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808039 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808042 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808045 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808048 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808050 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808053 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808056 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808059 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808061 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808064 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808067 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808069 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808072 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808075 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808077 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:24:50.069409 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808080 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808082 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808085 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808088 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808091 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808094 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808096 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808099 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808102 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808104 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808108 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808113 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808117 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808119 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808122 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808125 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808128 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808131 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808133 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:24:50.070765 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808137 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808141 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808143 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808146 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808149 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808151 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808154 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808156 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808159 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808161 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808164 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808167 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808169 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808172 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808174 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808177 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808180 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808183 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808185 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808191 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:24:50.071588 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808194 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808196 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808199 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808201 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808204 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808208 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808211 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808214 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808217 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808221 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808224 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808227 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808231 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808235 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808238 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808240 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808243 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808246 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808249 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:24:50.072441 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808251 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808254 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808257 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808259 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808262 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808264 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808267 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808269 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808272 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808275 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808277 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808281 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808285 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808288 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808290 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808702 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808707 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808710 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808713 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808715 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:24:50.073256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808718 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808721 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808724 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808727 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808730 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808733 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808737 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808740 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808744 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808747 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808750 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808752 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808756 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808758 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808761 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808764 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808766 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808769 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808771 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:24:50.076665 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808774 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808776 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808779 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808781 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808785 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808790 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808793 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808795 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808798 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808800 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808803 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808805 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808808 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808810 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808813 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808815 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808818 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808823 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808826 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:24:50.090892 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808828 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808831 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808833 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808836 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808840 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808844 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808847 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808850 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808853 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808856 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808859 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808862 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808864 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808867 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808869 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808872 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808874 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808879 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808884 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:24:50.093149 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808887 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808889 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808892 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808894 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808897 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808899 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808920 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808925 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808928 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808930 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808933 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808936 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808940 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808942 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808945 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808947 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808950 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808952 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808956 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808959 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:24:50.095338 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808961 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808964 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808966 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808969 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808971 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808974 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808976 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808979 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808981 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808984 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808988 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808992 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808995 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.808997 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809000 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809003 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809005 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809082 2572 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809091 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809097 2572 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809102 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 19:24:50.097687 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809106 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809110 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809114 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809120 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809126 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809129 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809133 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809136 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809139 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809143 2572 flags.go:64] FLAG: --cgroup-root="" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809146 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809149 2572 flags.go:64] FLAG: --client-ca-file="" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809151 2572 flags.go:64] FLAG: --cloud-provider="external" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809154 2572 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809159 2572 flags.go:64] FLAG: --cluster-domain="" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809162 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809165 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809168 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809171 2572 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809176 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809179 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809184 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809188 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809191 2572 flags.go:64] FLAG: --contention-profiling="false" Jul 06 19:24:50.100428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809194 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809197 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809201 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809203 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809208 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809211 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809213 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809216 2572 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809219 2572 flags.go:64] FLAG: --enable-server="true" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809222 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809226 2572 flags.go:64] FLAG: --event-burst="100" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809230 2572 flags.go:64] FLAG: --event-qps="50" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809233 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809236 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809239 2572 flags.go:64] FLAG: --eviction-hard="" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809243 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809246 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809250 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809253 2572 flags.go:64] FLAG: --eviction-soft="" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809256 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809259 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809262 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809265 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809268 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809271 2572 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 19:24:50.101578 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809274 2572 flags.go:64] FLAG: --feature-gates="" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809278 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809281 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809284 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809289 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809294 2572 flags.go:64] FLAG: --healthz-port="10248" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809297 2572 flags.go:64] FLAG: --help="false" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809300 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-131-35.ec2.internal" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809303 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809306 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809309 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809312 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809316 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809319 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809321 2572 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809324 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809327 2572 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809330 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809334 2572 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809336 2572 flags.go:64] FLAG: --kube-reserved="" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809339 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809343 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809346 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809349 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 19:24:50.102690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809352 2572 flags.go:64] FLAG: --lock-file="" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809355 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809358 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809361 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809366 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809369 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809372 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809375 2572 flags.go:64] FLAG: --logging-format="text" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809378 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809382 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809384 2572 flags.go:64] FLAG: --manifest-url="" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809387 2572 flags.go:64] FLAG: --manifest-url-header="" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809394 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809398 2572 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809403 2572 flags.go:64] FLAG: --max-pods="110" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809406 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809409 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809412 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809415 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809418 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809421 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809423 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809431 2572 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809434 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809437 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 19:24:50.103507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809440 2572 flags.go:64] FLAG: --pod-cidr="" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809443 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809449 2572 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809452 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809456 2572 flags.go:64] FLAG: --pods-per-core="0" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809459 2572 flags.go:64] FLAG: --port="10250" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809462 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809465 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a9ca3c20afab5b13" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809469 2572 flags.go:64] FLAG: --qos-reserved="" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809471 2572 flags.go:64] FLAG: --read-only-port="10255" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809474 2572 flags.go:64] FLAG: --register-node="true" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809477 2572 flags.go:64] FLAG: --register-with-taints="" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809481 2572 flags.go:64] FLAG: --registry-burst="10" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809483 2572 flags.go:64] FLAG: --registry-qps="5" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809486 2572 flags.go:64] FLAG: --reserved-cpus="" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809489 2572 flags.go:64] FLAG: --reserved-memory="" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809493 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809496 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809500 2572 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809504 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809507 2572 flags.go:64] FLAG: --runonce="false" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809511 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809515 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809518 2572 flags.go:64] FLAG: --seccomp-default="false" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809521 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 19:24:50.106584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809523 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809527 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809530 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809533 2572 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809536 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809538 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809541 2572 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809544 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809547 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809550 2572 flags.go:64] FLAG: --system-cgroups="" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809553 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809559 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809562 2572 flags.go:64] FLAG: --tls-cert-file="" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809566 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809571 2572 flags.go:64] FLAG: --tls-min-version="" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809573 2572 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809576 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809579 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809582 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809585 2572 flags.go:64] FLAG: --v="2" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809589 2572 flags.go:64] FLAG: --version="false" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809594 2572 flags.go:64] FLAG: --vmodule="" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809597 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.809600 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809747 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:24:50.107394 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809751 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809756 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809759 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809764 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809768 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809772 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809775 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809778 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809781 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809783 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809786 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809789 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809791 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809794 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809796 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809799 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809802 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809805 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809808 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:24:50.108891 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809810 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809813 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809816 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809819 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809822 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809824 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809827 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809830 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809832 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809836 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809838 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809841 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809843 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809846 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809850 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809853 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809857 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809860 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809862 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809865 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:24:50.109474 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809867 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809870 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809873 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809875 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809878 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809880 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809884 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809887 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809890 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809893 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809896 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809899 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809916 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809920 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809923 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809935 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809937 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809941 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809945 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:24:50.110024 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809948 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809951 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809954 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809956 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809959 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809962 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809964 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809969 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809972 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809975 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809978 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809981 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809984 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809986 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809989 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809992 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809994 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809997 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.809999 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810002 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:24:50.250094 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810004 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810007 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810009 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810012 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810014 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810017 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810019 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810022 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810025 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810034 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810037 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810039 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810042 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810045 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810048 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810050 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810053 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810055 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810058 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:24:50.251050 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.810062 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.810792 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.818387 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.818411 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.819501 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.819511 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819633 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819642 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819646 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819650 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819654 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819658 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819662 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819666 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:24:50.251731 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819670 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819673 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819677 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819681 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819685 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819689 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819696 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819701 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819708 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819713 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819718 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819722 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819726 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819730 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819733 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819738 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819743 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819746 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819750 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:24:50.252438 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819754 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819757 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819770 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819774 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819778 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819782 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819786 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819790 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819794 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819797 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819801 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819806 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819809 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819813 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819816 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819820 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819824 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819828 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819832 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819836 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:24:50.253139 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819840 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819843 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819848 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819852 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819856 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819860 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819863 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819869 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819873 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819877 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819881 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819885 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819889 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819893 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819897 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819938 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819944 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819948 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819952 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:24:50.253835 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819956 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819960 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819964 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819968 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819971 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819973 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819976 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819979 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819981 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819984 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819986 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819989 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819991 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819994 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.819997 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820000 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820003 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820005 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820008 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820011 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:24:50.254559 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820013 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820016 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820019 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820022 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820025 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820028 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820030 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820033 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820039 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820048 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820051 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820054 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820057 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.820062 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820240 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:24:50.255274 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820249 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820252 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820256 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820260 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820263 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820267 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820270 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820274 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820277 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820281 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820284 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820287 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820291 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820294 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820298 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820301 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820304 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820308 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820311 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820314 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:24:50.255991 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820317 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820322 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820326 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820329 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820333 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820338 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820341 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820354 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820358 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820364 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820370 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820375 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820380 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820385 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820389 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820393 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820397 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820401 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820406 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:24:50.256674 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820411 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820415 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820419 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820423 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820426 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820430 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820434 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820438 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820442 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820446 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820449 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820453 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820456 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820460 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820464 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820468 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820472 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820475 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820481 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820487 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:24:50.257373 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820492 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820505 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820511 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820515 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820519 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820523 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820527 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820530 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820534 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820537 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820541 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820545 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820549 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820553 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820557 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820560 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820564 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820567 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820571 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820575 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:24:50.258087 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820579 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820582 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820586 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820590 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820594 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820598 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820602 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820605 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820609 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820613 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820618 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820622 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820625 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820629 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820642 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820647 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820651 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820654 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:24:50.258775 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:49.820657 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.820665 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.820875 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.825310 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.826796 2572 server.go:1019] "Starting client certificate rotation" Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.826899 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.826949 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.829481 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.830715 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.840484 2572 log.go:25] "Validated CRI v1 runtime API" Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.842486 2572 log.go:25] "Validated CRI v1 image API" Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.843271 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 19:24:50.259488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.846802 2572 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c5e0bf50-b34e-472a-87ad-0d974ba0e447:/dev/nvme0n1p4 f454d41e-055e-4442-b8ef-3ef9cd39a023:/dev/nvme0n1p3] Jul 06 19:24:50.260211 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.846818 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 19:24:50.260550 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.853797 2572 manager.go:216] Machine: {Timestamp:2026-07-06 19:24:49.852685405 +0000 UTC m=+0.357970434 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099414 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20b66427fab71e5abb0155fe657b78 SystemUUID:ec20b664-27fa-b71e-5abb-0155fe657b78 BootID:715fde44-ad04-4868-8adf-7bd98b407b04 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:59:34:9f:43:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:59:34:9f:43:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:a2:ad:b1:c8:f3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 19:24:50.260550 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.853925 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 19:24:50.260550 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.854017 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 19:24:50.260550 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.855208 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 19:24:50.260550 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.855248 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-35.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 19:24:50.260550 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.855980 2572 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 19:24:50.260550 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.855991 2572 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 19:24:50.260550 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.855999 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.856569 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.857895 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.859483 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.860804 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.860948 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.860975 2572 kubelet.go:486] "Attempting to sync node with API server" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.860990 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.861008 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.861017 2572 kubelet.go:398] "Adding apiserver pod source" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.861025 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.861499 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.861530 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.865314 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.871203 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.871232 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872314 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872329 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872335 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872341 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872350 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872355 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872364 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872369 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872379 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872385 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872393 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.872399 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.873064 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.873071 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.874684 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.874729 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.876306 2572 server.go:1268] "Started kubelet" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.876401 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.876421 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.876509 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.877556 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.878367 2572 server.go:310] "Adding debug handlers to kubelet server" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.883239 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 19:24:50.260917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.885013 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-35.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.885629 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.885633 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.885746 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-35.ec2.internal\" not found" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.885027 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-35.ec2.internal.18bfc9d4029fb38d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-35.ec2.internal,UID:ip-10-0-131-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-35.ec2.internal,},FirstTimestamp:2026-07-06 19:24:49.876276109 +0000 UTC m=+0.381561187,LastTimestamp:2026-07-06 19:24:49.876276109 +0000 UTC m=+0.381561187,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-35.ec2.internal,}" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.887023 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.887037 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.887028 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.887399 2572 reconstruct.go:97] "Volume reconstruction finished" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.887405 2572 reconciler.go:29] "Reconciler: start to sync state" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.887642 2572 factory.go:55] Registering systemd factory Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.887659 2572 factory.go:223] Registration of the systemd container factory successfully Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.888941 2572 factory.go:153] Registering CRI-O factory Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.888961 2572 factory.go:223] Registration of the crio container factory successfully Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.889027 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.889050 2572 factory.go:103] Registering Raw factory Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.889064 2572 manager.go:1196] Started watching for new ooms in manager Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.889494 2572 manager.go:349] Starting recovery of all containers Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.893025 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.893133 2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.898422 2572 manager.go:354] Recovery completed Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.909791 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.911715 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.911749 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.911760 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.912335 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.912344 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.912359 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.912408 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.913129 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.913155 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.913165 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.913522 2572 policy_none.go:49] "None policy: Start" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.913544 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 19:24:50.266641 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.913559 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.913607 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-35.ec2.internal.18bfc9d404bcbd47 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-35.ec2.internal,UID:ip-10-0-131-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-35.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-35.ec2.internal,},FirstTimestamp:2026-07-06 19:24:49.911733575 +0000 UTC m=+0.417018607,LastTimestamp:2026-07-06 19:24:49.911733575 +0000 UTC m=+0.417018607,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-35.ec2.internal,}" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.914613 2572 policy_none.go:47] "Start" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.915856 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x97q5" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.922147 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-35.ec2.internal.18bfc9d404bd0c11 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-35.ec2.internal,UID:ip-10-0-131-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-35.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-35.ec2.internal,},FirstTimestamp:2026-07-06 19:24:49.911753745 +0000 UTC m=+0.417038773,LastTimestamp:2026-07-06 19:24:49.911753745 +0000 UTC m=+0.417038773,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-35.ec2.internal,}" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.927161 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x97q5" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.933178 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-35.ec2.internal.18bfc9d404bd3383 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-35.ec2.internal,UID:ip-10-0-131-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-35.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-35.ec2.internal,},FirstTimestamp:2026-07-06 19:24:49.911763843 +0000 UTC m=+0.417048872,LastTimestamp:2026-07-06 19:24:49.911763843 +0000 UTC m=+0.417048872,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-35.ec2.internal,}" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.953540 2572 manager.go:337] "Starting Device Plugin manager" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.953578 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.953589 2572 server.go:87] "Starting device plugin registration server" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.953872 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.953885 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.953987 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.954076 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:49.954085 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.954628 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:49.954675 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-35.ec2.internal\" not found" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.054214 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.055215 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:24:50.267698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.055253 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.055269 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.055305 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-35.ec2.internal" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.065457 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-35.ec2.internal" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:50.065486 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-35.ec2.internal\": node \"ip-10-0-131-35.ec2.internal\" not found" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:50.091417 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-35.ec2.internal\" not found" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.161901 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.168733 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.168771 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.168799 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:50.168838 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.170790 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:50.191881 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-35.ec2.internal\" not found" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.269090 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-35.ec2.internal"] Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.269184 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.273070 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.273096 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.273105 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.274262 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.274683 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.274707 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.275442 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.275461 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.275473 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.275691 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.275706 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.275718 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.276875 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-35.ec2.internal" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.276895 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.277874 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.277895 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.277963 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:24:50.363243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.289779 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5dcde92490703d276672775cee39ea5c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal\" (UID: \"5dcde92490703d276672775cee39ea5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" Jul 06 19:24:50.364297 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.289803 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dcde92490703d276672775cee39ea5c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal\" (UID: \"5dcde92490703d276672775cee39ea5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" Jul 06 19:24:50.364297 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.289823 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a91a71edc512a42ccb4fd1df84711f15-config\") pod \"kube-apiserver-proxy-ip-10-0-131-35.ec2.internal\" (UID: \"a91a71edc512a42ccb4fd1df84711f15\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-35.ec2.internal" Jul 06 19:24:50.364297 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:50.292634 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-35.ec2.internal\" not found" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.905374 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:50.905374 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-35.ec2.internal\" not found" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.905437 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5dcde92490703d276672775cee39ea5c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal\" (UID: \"5dcde92490703d276672775cee39ea5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.905470 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dcde92490703d276672775cee39ea5c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal\" (UID: \"5dcde92490703d276672775cee39ea5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.905489 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a91a71edc512a42ccb4fd1df84711f15-config\") pod \"kube-apiserver-proxy-ip-10-0-131-35.ec2.internal\" (UID: \"a91a71edc512a42ccb4fd1df84711f15\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.905550 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a91a71edc512a42ccb4fd1df84711f15-config\") pod \"kube-apiserver-proxy-ip-10-0-131-35.ec2.internal\" (UID: \"a91a71edc512a42ccb4fd1df84711f15\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.905557 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5dcde92490703d276672775cee39ea5c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal\" (UID: \"5dcde92490703d276672775cee39ea5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.905569 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.905550 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dcde92490703d276672775cee39ea5c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal\" (UID: \"5dcde92490703d276672775cee39ea5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:50.930125 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-35.ec2.internal\" not found" node="ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.931825 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 19:19:50 +0000 UTC" deadline="2028-02-13 08:39:47.575534402 +0000 UTC" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.931850 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14077h14m56.64368732s" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.932419 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:50.935290 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-35.ec2.internal\" not found" node="ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:50.938024 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:51.006156 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-35.ec2.internal\" not found" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.021450 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.022589 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.022627 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:24:51.025293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.022638 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:24:51.137850 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.103382 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:24:51.137850 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:51.107011 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-35.ec2.internal\" not found" Jul 06 19:24:51.249531 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:51.207690 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-35.ec2.internal\" not found" Jul 06 19:24:51.370553 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:51.308620 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-35.ec2.internal\" not found" Jul 06 19:24:51.388439 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.385194 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:24:51.388439 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.386097 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.488957 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.405324 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:24:51.488957 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.406421 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-35.ec2.internal" Jul 06 19:24:51.488957 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.414287 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:24:51.886391 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.886356 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 19:24:51.905450 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.905420 2572 apiserver.go:52] "Watching apiserver" Jul 06 19:24:51.905645 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.905621 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:24:51.907562 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.907547 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:24:51.912704 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.912680 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 19:24:51.914010 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.913962 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-cjr7k","kube-system/konnectivity-agent-5pnv2","openshift-cluster-node-tuning-operator/tuned-dhd89","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal","openshift-multus/multus-additional-cni-plugins-zqg6b","openshift-multus/network-metrics-daemon-5942g","openshift-network-diagnostics/network-check-target-ltflt","openshift-ovn-kubernetes/ovnkube-node-5d4pt","kube-system/kube-apiserver-proxy-ip-10-0-131-35.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d","openshift-dns/node-resolver-ljx8t","openshift-image-registry/node-ca-z2r5s","openshift-multus/multus-9crmz","openshift-network-operator/iptables-alerter-7w9f7"] Jul 06 19:24:51.915546 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.915526 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:51.917222 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.917197 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9crmz" Jul 06 19:24:51.918280 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.918260 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 19:24:51.918384 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.918259 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 19:24:51.918384 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.918370 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 19:24:51.918505 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.918379 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lwhnc\"" Jul 06 19:24:51.918593 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.918578 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:51.919647 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.919629 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9ssp8\"" Jul 06 19:24:51.919742 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.919632 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 19:24:51.919742 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.919735 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 19:24:51.919991 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.919961 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 19:24:51.920103 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.920069 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7w9f7" Jul 06 19:24:51.920718 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.920702 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x7g9w\"" Jul 06 19:24:51.920769 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.920722 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:24:51.921070 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.921054 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 19:24:51.921135 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.921098 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 19:24:51.921433 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.921411 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:51.922267 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.922248 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t8qqm\"" Jul 06 19:24:51.922389 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.922371 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:24:51.922717 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.922697 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:51.922819 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.922727 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 19:24:51.922819 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.922729 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 19:24:51.922931 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:51.922814 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:24:51.923617 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.923599 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 19:24:51.923699 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.923687 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7rfxj\"" Jul 06 19:24:51.923772 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.923755 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 19:24:51.924324 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.924308 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:51.926203 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.926186 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:24:51.926283 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:51.926242 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:24:51.926485 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.926472 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-m7hqr" Jul 06 19:24:51.926547 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.926537 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gjwv8\"" Jul 06 19:24:51.926610 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.926594 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 19:24:51.927157 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.927133 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 19:24:51.929539 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.927296 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 19:24:51.929767 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.929625 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 19:24:51.929767 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.929717 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:24:51.929877 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.929835 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 19:24:51.930283 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.930114 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 19:24:51.932339 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.932317 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 19:24:51.932437 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.932354 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 19:24:51.932536 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.932519 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fc8qn\"" Jul 06 19:24:51.932773 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.932757 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:51.932865 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:51.932842 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:24:51.934371 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.934349 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ljx8t" Jul 06 19:24:51.935399 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.935379 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-m7hqr" Jul 06 19:24:51.935958 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.935902 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z2r5s" Jul 06 19:24:51.936612 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.936591 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 19:24:51.936727 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.936707 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-89757\"" Jul 06 19:24:51.936727 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.936717 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 19:24:51.938105 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.938091 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 19:24:51.938238 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.938225 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 19:24:51.938465 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.938451 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 19:24:51.938649 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.938636 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xg6kd\"" Jul 06 19:24:51.972672 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:51.972579 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcde92490703d276672775cee39ea5c.slice/crio-93bfaa747643603046fbc1de204eb13796699af18553d1c04b11bdefb22a8614 WatchSource:0}: Error finding container 93bfaa747643603046fbc1de204eb13796699af18553d1c04b11bdefb22a8614: Status 404 returned error can't find the container with id 93bfaa747643603046fbc1de204eb13796699af18553d1c04b11bdefb22a8614 Jul 06 19:24:51.972861 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:51.972829 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91a71edc512a42ccb4fd1df84711f15.slice/crio-e844631cd969e249125290b6f4758fcd49fa139836847f96ff6b80403611a9de WatchSource:0}: Error finding container e844631cd969e249125290b6f4758fcd49fa139836847f96ff6b80403611a9de: Status 404 returned error can't find the container with id e844631cd969e249125290b6f4758fcd49fa139836847f96ff6b80403611a9de Jul 06 19:24:51.979949 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.979926 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:24:51.988325 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:51.988292 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 19:24:52.013683 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.013643 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-log-socket\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.013834 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.013699 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-daemon-config\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.013834 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.013727 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68f9a6e3-08f5-4aa1-aba6-f07ff314e006-hosts-file\") pod \"node-resolver-ljx8t\" (UID: \"68f9a6e3-08f5-4aa1-aba6-f07ff314e006\") " pod="openshift-dns/node-resolver-ljx8t" Jul 06 19:24:52.013834 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.013807 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-var-lib-cni-bin\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.014022 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.013847 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-var-lib-kubelet\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.014022 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.013876 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-systemd-units\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.014022 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.013901 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-run-netns\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.014022 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.013944 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-cni-dir\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.014022 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.013968 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-cni-binary-copy\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.014022 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.013991 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-run-multus-certs\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.014322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014039 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.014322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014081 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9621c68e-6ddf-4ff6-a451-062347974270-iptables-alerter-script\") pod \"iptables-alerter-7w9f7\" (UID: \"9621c68e-6ddf-4ff6-a451-062347974270\") " pod="openshift-network-operator/iptables-alerter-7w9f7" Jul 06 19:24:52.014322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014104 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-os-release\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.014322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014131 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr4mk\" (UniqueName: \"kubernetes.io/projected/8614ab98-8436-487f-8a3b-4485b54c043f-kube-api-access-hr4mk\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.014322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014153 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-kubernetes\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.014322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014195 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-host\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.014322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014243 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cbf09284-56b5-453a-8ed0-c33a09b04265-env-overrides\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.014322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014272 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09767752-33c4-4574-b852-5d40a6c8929f-host\") pod \"node-ca-z2r5s\" (UID: \"09767752-33c4-4574-b852-5d40a6c8929f\") " pod="openshift-image-registry/node-ca-z2r5s" Jul 06 19:24:52.014322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014297 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-system-cni-dir\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.014322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014322 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-var-lib-cni-multus\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014358 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014400 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-registration-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014421 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-run\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014438 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-dbus\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014460 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dldkr\" (UniqueName: \"kubernetes.io/projected/ecb7b845-f496-49e7-87e3-a47752a5944b-kube-api-access-dldkr\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014491 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmz55\" (UniqueName: \"kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55\") pod \"network-check-target-ltflt\" (UID: \"8acd900e-9116-4692-a959-3ecf8035a515\") " pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014514 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-modprobe-d\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014553 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-sys\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014571 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-cnibin\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014597 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-cni-netd\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014623 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68f9a6e3-08f5-4aa1-aba6-f07ff314e006-tmp-dir\") pod \"node-resolver-ljx8t\" (UID: \"68f9a6e3-08f5-4aa1-aba6-f07ff314e006\") " pod="openshift-dns/node-resolver-ljx8t" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014640 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-sys-fs\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014656 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-sysctl-d\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.014843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014673 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-systemd\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014720 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpzc2\" (UniqueName: \"kubernetes.io/projected/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-kube-api-access-rpzc2\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014746 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-node-log\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014776 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/663a3e6c-13db-45ac-b7fd-0efd27a4b159-agent-certs\") pod \"konnectivity-agent-5pnv2\" (UID: \"663a3e6c-13db-45ac-b7fd-0efd27a4b159\") " pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014796 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/663a3e6c-13db-45ac-b7fd-0efd27a4b159-konnectivity-ca\") pod \"konnectivity-agent-5pnv2\" (UID: \"663a3e6c-13db-45ac-b7fd-0efd27a4b159\") " pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014816 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014844 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-etc-selinux\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014878 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-run-ovn-kubernetes\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014902 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz9j4\" (UniqueName: \"kubernetes.io/projected/09767752-33c4-4574-b852-5d40a6c8929f-kube-api-access-bz9j4\") pod \"node-ca-z2r5s\" (UID: \"09767752-33c4-4574-b852-5d40a6c8929f\") " pod="openshift-image-registry/node-ca-z2r5s" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014940 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-socket-dir-parent\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.014996 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-run-netns\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015021 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9dj6\" (UniqueName: \"kubernetes.io/projected/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-kube-api-access-h9dj6\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015059 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-os-release\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.015658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015099 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015126 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cbf09284-56b5-453a-8ed0-c33a09b04265-ovnkube-config\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015144 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cbf09284-56b5-453a-8ed0-c33a09b04265-ovnkube-script-lib\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015174 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015198 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-kubelet\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015522 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cbf09284-56b5-453a-8ed0-c33a09b04265-ovn-node-metrics-cert\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015580 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msdlp\" (UniqueName: \"kubernetes.io/projected/cbf09284-56b5-453a-8ed0-c33a09b04265-kube-api-access-msdlp\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015624 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-system-cni-dir\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015660 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015695 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-var-lib-openvswitch\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015731 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-run-openvswitch\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015777 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-cni-bin\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015806 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-kubelet-config\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:52.016282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015854 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-hostroot\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.015889 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-conf-dir\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016051 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njm7v\" (UniqueName: \"kubernetes.io/projected/9621c68e-6ddf-4ff6-a451-062347974270-kube-api-access-njm7v\") pod \"iptables-alerter-7w9f7\" (UID: \"9621c68e-6ddf-4ff6-a451-062347974270\") " pod="openshift-network-operator/iptables-alerter-7w9f7" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016103 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b19f7e0-96e8-45b3-8b56-eb3651786216-tmp\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016143 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjj8p\" (UniqueName: \"kubernetes.io/projected/8b19f7e0-96e8-45b3-8b56-eb3651786216-kube-api-access-jjj8p\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016172 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9621c68e-6ddf-4ff6-a451-062347974270-host-slash\") pod \"iptables-alerter-7w9f7\" (UID: \"9621c68e-6ddf-4ff6-a451-062347974270\") " pod="openshift-network-operator/iptables-alerter-7w9f7" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016220 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-run-k8s-cni-cncf-io\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016277 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-etc-kubernetes\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016366 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-sysctl-conf\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016411 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-cni-binary-copy\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016449 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-etc-openvswitch\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016507 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-run-ovn\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016537 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09767752-33c4-4574-b852-5d40a6c8929f-serviceca\") pod \"node-ca-z2r5s\" (UID: \"09767752-33c4-4574-b852-5d40a6c8929f\") " pod="openshift-image-registry/node-ca-z2r5s" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016571 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-device-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.016967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016605 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.017654 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016656 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-run-systemd\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.017654 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016698 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-cnibin\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.017654 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016730 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n852\" (UniqueName: \"kubernetes.io/projected/68f9a6e3-08f5-4aa1-aba6-f07ff314e006-kube-api-access-7n852\") pod \"node-resolver-ljx8t\" (UID: \"68f9a6e3-08f5-4aa1-aba6-f07ff314e006\") " pod="openshift-dns/node-resolver-ljx8t" Jul 06 19:24:52.017654 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016764 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-socket-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.017654 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016799 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-sysconfig\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.017654 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016831 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-lib-modules\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.017654 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016859 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-var-lib-kubelet\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.017654 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016894 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-tuned\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.017654 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.016926 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-slash\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.117509 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117471 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjj8p\" (UniqueName: \"kubernetes.io/projected/8b19f7e0-96e8-45b3-8b56-eb3651786216-kube-api-access-jjj8p\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.117509 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117515 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9621c68e-6ddf-4ff6-a451-062347974270-host-slash\") pod \"iptables-alerter-7w9f7\" (UID: \"9621c68e-6ddf-4ff6-a451-062347974270\") " pod="openshift-network-operator/iptables-alerter-7w9f7" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117533 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-run-k8s-cni-cncf-io\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117553 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-etc-kubernetes\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117577 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-sysctl-conf\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117602 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-cni-binary-copy\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117610 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9621c68e-6ddf-4ff6-a451-062347974270-host-slash\") pod \"iptables-alerter-7w9f7\" (UID: \"9621c68e-6ddf-4ff6-a451-062347974270\") " pod="openshift-network-operator/iptables-alerter-7w9f7" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117626 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-run-k8s-cni-cncf-io\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117641 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-etc-kubernetes\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117682 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-etc-openvswitch\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117728 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-run-ovn\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117748 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-sysctl-conf\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117754 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09767752-33c4-4574-b852-5d40a6c8929f-serviceca\") pod \"node-ca-z2r5s\" (UID: \"09767752-33c4-4574-b852-5d40a6c8929f\") " pod="openshift-image-registry/node-ca-z2r5s" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117791 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-device-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117765 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-etc-openvswitch\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117811 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-run-ovn\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.118269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117814 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117859 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-run-systemd\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117883 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-cnibin\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117894 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-device-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117911 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n852\" (UniqueName: \"kubernetes.io/projected/68f9a6e3-08f5-4aa1-aba6-f07ff314e006-kube-api-access-7n852\") pod \"node-resolver-ljx8t\" (UID: \"68f9a6e3-08f5-4aa1-aba6-f07ff314e006\") " pod="openshift-dns/node-resolver-ljx8t" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117957 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-socket-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117962 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-run-systemd\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.117982 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-sysconfig\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118006 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-lib-modules\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118017 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118031 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-var-lib-kubelet\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118058 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-tuned\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118073 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-cnibin\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118081 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-slash\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118103 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-log-socket\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.119023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118125 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-daemon-config\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118147 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68f9a6e3-08f5-4aa1-aba6-f07ff314e006-hosts-file\") pod \"node-resolver-ljx8t\" (UID: \"68f9a6e3-08f5-4aa1-aba6-f07ff314e006\") " pod="openshift-dns/node-resolver-ljx8t" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118172 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-var-lib-cni-bin\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118192 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-lib-modules\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118202 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09767752-33c4-4574-b852-5d40a6c8929f-serviceca\") pod \"node-ca-z2r5s\" (UID: \"09767752-33c4-4574-b852-5d40a6c8929f\") " pod="openshift-image-registry/node-ca-z2r5s" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118221 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-var-lib-kubelet\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118224 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-cni-binary-copy\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118252 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-sysconfig\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118276 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-var-lib-kubelet\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118282 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-var-lib-cni-bin\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118296 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-systemd-units\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118284 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68f9a6e3-08f5-4aa1-aba6-f07ff314e006-hosts-file\") pod \"node-resolver-ljx8t\" (UID: \"68f9a6e3-08f5-4aa1-aba6-f07ff314e006\") " pod="openshift-dns/node-resolver-ljx8t" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118227 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-slash\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118316 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-run-netns\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118333 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-cni-dir\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.119768 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118347 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-socket-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118353 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-systemd-units\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118406 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-log-socket\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118350 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-cni-binary-copy\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118450 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-run-multus-certs\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118467 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118483 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9621c68e-6ddf-4ff6-a451-062347974270-iptables-alerter-script\") pod \"iptables-alerter-7w9f7\" (UID: \"9621c68e-6ddf-4ff6-a451-062347974270\") " pod="openshift-network-operator/iptables-alerter-7w9f7" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118499 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-os-release\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118542 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-run-netns\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118544 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118560 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118551 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-cni-dir\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118565 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-var-lib-kubelet\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118607 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-os-release\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118614 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr4mk\" (UniqueName: \"kubernetes.io/projected/8614ab98-8436-487f-8a3b-4485b54c043f-kube-api-access-hr4mk\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.120511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118643 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-kubernetes\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118648 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-run-multus-certs\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118667 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-host\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118708 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cbf09284-56b5-453a-8ed0-c33a09b04265-env-overrides\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118724 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-host\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118740 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09767752-33c4-4574-b852-5d40a6c8929f-host\") pod \"node-ca-z2r5s\" (UID: \"09767752-33c4-4574-b852-5d40a6c8929f\") " pod="openshift-image-registry/node-ca-z2r5s" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118769 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-system-cni-dir\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118799 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-var-lib-cni-multus\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118803 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-kubernetes\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118829 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118858 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-registration-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118869 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-cni-binary-copy\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118895 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-var-lib-cni-multus\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118832 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-daemon-config\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118914 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-run\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.121305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118936 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-system-cni-dir\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118853 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09767752-33c4-4574-b852-5d40a6c8929f-host\") pod \"node-ca-z2r5s\" (UID: \"09767752-33c4-4574-b852-5d40a6c8929f\") " pod="openshift-image-registry/node-ca-z2r5s" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118961 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-dbus\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118988 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dldkr\" (UniqueName: \"kubernetes.io/projected/ecb7b845-f496-49e7-87e3-a47752a5944b-kube-api-access-dldkr\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.118997 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-registration-dir\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119015 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-run\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.119022 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.119122 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs podName:ecb7b845-f496-49e7-87e3-a47752a5944b nodeName:}" failed. No retries permitted until 2026-07-06 19:24:52.619080498 +0000 UTC m=+3.124365531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs") pod "network-metrics-daemon-5942g" (UID: "ecb7b845-f496-49e7-87e3-a47752a5944b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119154 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-dbus\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119018 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmz55\" (UniqueName: \"kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55\") pod \"network-check-target-ltflt\" (UID: \"8acd900e-9116-4692-a959-3ecf8035a515\") " pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119180 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9621c68e-6ddf-4ff6-a451-062347974270-iptables-alerter-script\") pod \"iptables-alerter-7w9f7\" (UID: \"9621c68e-6ddf-4ff6-a451-062347974270\") " pod="openshift-network-operator/iptables-alerter-7w9f7" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119205 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-modprobe-d\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119245 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-sys\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119262 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cbf09284-56b5-453a-8ed0-c33a09b04265-env-overrides\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.121797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119297 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-cnibin\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119316 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-modprobe-d\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119330 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-sys\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119332 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-cnibin\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119335 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-cni-netd\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119366 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-cni-netd\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119376 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68f9a6e3-08f5-4aa1-aba6-f07ff314e006-tmp-dir\") pod \"node-resolver-ljx8t\" (UID: \"68f9a6e3-08f5-4aa1-aba6-f07ff314e006\") " pod="openshift-dns/node-resolver-ljx8t" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119394 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-sys-fs\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119409 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-sysctl-d\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119464 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-sys-fs\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119488 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-sysctl-d\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119499 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-systemd\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119529 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpzc2\" (UniqueName: \"kubernetes.io/projected/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-kube-api-access-rpzc2\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119560 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-node-log\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119587 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/663a3e6c-13db-45ac-b7fd-0efd27a4b159-agent-certs\") pod \"konnectivity-agent-5pnv2\" (UID: \"663a3e6c-13db-45ac-b7fd-0efd27a4b159\") " pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:24:52.122268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119603 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-systemd\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119613 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-node-log\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119615 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/663a3e6c-13db-45ac-b7fd-0efd27a4b159-konnectivity-ca\") pod \"konnectivity-agent-5pnv2\" (UID: \"663a3e6c-13db-45ac-b7fd-0efd27a4b159\") " pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119643 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119647 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68f9a6e3-08f5-4aa1-aba6-f07ff314e006-tmp-dir\") pod \"node-resolver-ljx8t\" (UID: \"68f9a6e3-08f5-4aa1-aba6-f07ff314e006\") " pod="openshift-dns/node-resolver-ljx8t" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119671 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-etc-selinux\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119700 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-run-ovn-kubernetes\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119748 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz9j4\" (UniqueName: \"kubernetes.io/projected/09767752-33c4-4574-b852-5d40a6c8929f-kube-api-access-bz9j4\") pod \"node-ca-z2r5s\" (UID: \"09767752-33c4-4574-b852-5d40a6c8929f\") " pod="openshift-image-registry/node-ca-z2r5s" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119751 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8614ab98-8436-487f-8a3b-4485b54c043f-etc-selinux\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119773 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-socket-dir-parent\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119760 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-run-ovn-kubernetes\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119825 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-run-netns\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119839 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-socket-dir-parent\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.119755 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119870 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dj6\" (UniqueName: \"kubernetes.io/projected/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-kube-api-access-h9dj6\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.122783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119879 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-host-run-netns\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.119958 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret podName:b2cbb8d1-a657-43ca-a8d1-bc09a049a710 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:52.619938661 +0000 UTC m=+3.125223679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret") pod "global-pull-secret-syncer-cjr7k" (UID: "b2cbb8d1-a657-43ca-a8d1-bc09a049a710") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.119979 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-os-release\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120012 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120045 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cbf09284-56b5-453a-8ed0-c33a09b04265-ovnkube-config\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120071 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cbf09284-56b5-453a-8ed0-c33a09b04265-ovnkube-script-lib\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120082 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120089 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-os-release\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120098 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120155 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-kubelet\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120188 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cbf09284-56b5-453a-8ed0-c33a09b04265-ovn-node-metrics-cert\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120214 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msdlp\" (UniqueName: \"kubernetes.io/projected/cbf09284-56b5-453a-8ed0-c33a09b04265-kube-api-access-msdlp\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120244 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-system-cni-dir\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.123329 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120274 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120304 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-var-lib-openvswitch\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120349 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-run-openvswitch\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120377 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-cni-bin\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120403 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-kubelet-config\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120430 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-hostroot\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120471 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-conf-dir\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120499 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njm7v\" (UniqueName: \"kubernetes.io/projected/9621c68e-6ddf-4ff6-a451-062347974270-kube-api-access-njm7v\") pod \"iptables-alerter-7w9f7\" (UID: \"9621c68e-6ddf-4ff6-a451-062347974270\") " pod="openshift-network-operator/iptables-alerter-7w9f7" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120525 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b19f7e0-96e8-45b3-8b56-eb3651786216-tmp\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120640 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120652 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/663a3e6c-13db-45ac-b7fd-0efd27a4b159-konnectivity-ca\") pod \"konnectivity-agent-5pnv2\" (UID: \"663a3e6c-13db-45ac-b7fd-0efd27a4b159\") " pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120733 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-hostroot\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120780 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-run-openvswitch\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120828 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-cni-bin\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120884 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-kubelet-config\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:52.123715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120934 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-system-cni-dir\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.124176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.120217 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-host-kubelet\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.124176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.121020 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-multus-conf-dir\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.124176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.121068 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbf09284-56b5-453a-8ed0-c33a09b04265-var-lib-openvswitch\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.124176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.121105 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cbf09284-56b5-453a-8ed0-c33a09b04265-ovnkube-config\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.124176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.121283 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cbf09284-56b5-453a-8ed0-c33a09b04265-ovnkube-script-lib\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.124176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.121516 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.124176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.122867 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b19f7e0-96e8-45b3-8b56-eb3651786216-etc-tuned\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.124176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.122895 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cbf09284-56b5-453a-8ed0-c33a09b04265-ovn-node-metrics-cert\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.124176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.122905 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b19f7e0-96e8-45b3-8b56-eb3651786216-tmp\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.124176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.123087 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/663a3e6c-13db-45ac-b7fd-0efd27a4b159-agent-certs\") pod \"konnectivity-agent-5pnv2\" (UID: \"663a3e6c-13db-45ac-b7fd-0efd27a4b159\") " pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:24:52.131702 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.131668 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:24:52.131702 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.131692 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:24:52.131702 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.131704 2572 projected.go:196] Error preparing data for projected volume kube-api-access-hmz55 for pod openshift-network-diagnostics/network-check-target-ltflt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:24:52.131952 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.131776 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55 podName:8acd900e-9116-4692-a959-3ecf8035a515 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:52.631750596 +0000 UTC m=+3.137035618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hmz55" (UniqueName: "kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55") pod "network-check-target-ltflt" (UID: "8acd900e-9116-4692-a959-3ecf8035a515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:24:52.132313 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.132291 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjj8p\" (UniqueName: \"kubernetes.io/projected/8b19f7e0-96e8-45b3-8b56-eb3651786216-kube-api-access-jjj8p\") pod \"tuned-dhd89\" (UID: \"8b19f7e0-96e8-45b3-8b56-eb3651786216\") " pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.134262 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.134236 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9dj6\" (UniqueName: \"kubernetes.io/projected/fcffc633-f4f2-4d7f-8fd2-d4c1819655dd-kube-api-access-h9dj6\") pod \"multus-9crmz\" (UID: \"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd\") " pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.134408 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.134383 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dldkr\" (UniqueName: \"kubernetes.io/projected/ecb7b845-f496-49e7-87e3-a47752a5944b-kube-api-access-dldkr\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:52.135365 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.135338 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpzc2\" (UniqueName: \"kubernetes.io/projected/e817f8b8-1d4a-445d-91c4-6c6d77d2bd28-kube-api-access-rpzc2\") pod \"multus-additional-cni-plugins-zqg6b\" (UID: \"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28\") " pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.135529 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.135511 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njm7v\" (UniqueName: \"kubernetes.io/projected/9621c68e-6ddf-4ff6-a451-062347974270-kube-api-access-njm7v\") pod \"iptables-alerter-7w9f7\" (UID: \"9621c68e-6ddf-4ff6-a451-062347974270\") " pod="openshift-network-operator/iptables-alerter-7w9f7" Jul 06 19:24:52.135801 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.135779 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz9j4\" (UniqueName: \"kubernetes.io/projected/09767752-33c4-4574-b852-5d40a6c8929f-kube-api-access-bz9j4\") pod \"node-ca-z2r5s\" (UID: \"09767752-33c4-4574-b852-5d40a6c8929f\") " pod="openshift-image-registry/node-ca-z2r5s" Jul 06 19:24:52.136257 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.136237 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n852\" (UniqueName: \"kubernetes.io/projected/68f9a6e3-08f5-4aa1-aba6-f07ff314e006-kube-api-access-7n852\") pod \"node-resolver-ljx8t\" (UID: \"68f9a6e3-08f5-4aa1-aba6-f07ff314e006\") " pod="openshift-dns/node-resolver-ljx8t" Jul 06 19:24:52.136649 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.136628 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr4mk\" (UniqueName: \"kubernetes.io/projected/8614ab98-8436-487f-8a3b-4485b54c043f-kube-api-access-hr4mk\") pod \"aws-ebs-csi-driver-node-zvt8d\" (UID: \"8614ab98-8436-487f-8a3b-4485b54c043f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.137169 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.137152 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msdlp\" (UniqueName: \"kubernetes.io/projected/cbf09284-56b5-453a-8ed0-c33a09b04265-kube-api-access-msdlp\") pod \"ovnkube-node-5d4pt\" (UID: \"cbf09284-56b5-453a-8ed0-c33a09b04265\") " pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.177071 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.177008 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-35.ec2.internal" event={"ID":"a91a71edc512a42ccb4fd1df84711f15","Type":"ContainerStarted","Data":"e844631cd969e249125290b6f4758fcd49fa139836847f96ff6b80403611a9de"} Jul 06 19:24:52.178128 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.178102 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" event={"ID":"5dcde92490703d276672775cee39ea5c","Type":"ContainerStarted","Data":"93bfaa747643603046fbc1de204eb13796699af18553d1c04b11bdefb22a8614"} Jul 06 19:24:52.232843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.232733 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" Jul 06 19:24:52.245333 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.245309 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9crmz" Jul 06 19:24:52.252513 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:52.252484 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcffc633_f4f2_4d7f_8fd2_d4c1819655dd.slice/crio-45e7c85a5abfabf840641fd9ef2b910913370a9d3f781c3cac4aa4026c570734 WatchSource:0}: Error finding container 45e7c85a5abfabf840641fd9ef2b910913370a9d3f781c3cac4aa4026c570734: Status 404 returned error can't find the container with id 45e7c85a5abfabf840641fd9ef2b910913370a9d3f781c3cac4aa4026c570734 Jul 06 19:24:52.281256 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.281219 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dhd89" Jul 06 19:24:52.288407 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.288387 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7w9f7" Jul 06 19:24:52.289604 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:52.289579 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b19f7e0_96e8_45b3_8b56_eb3651786216.slice/crio-a9db1aa95365f613394331229c3d7364f7186cfb8422f6d69dfe105a27a76caf WatchSource:0}: Error finding container a9db1aa95365f613394331229c3d7364f7186cfb8422f6d69dfe105a27a76caf: Status 404 returned error can't find the container with id a9db1aa95365f613394331229c3d7364f7186cfb8422f6d69dfe105a27a76caf Jul 06 19:24:52.293669 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.293608 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zqg6b" Jul 06 19:24:52.296140 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:52.296114 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9621c68e_6ddf_4ff6_a451_062347974270.slice/crio-af31e71138b166d2c1a99f986eb8aaea2290e2b1fece8b24de0807a31344363f WatchSource:0}: Error finding container af31e71138b166d2c1a99f986eb8aaea2290e2b1fece8b24de0807a31344363f: Status 404 returned error can't find the container with id af31e71138b166d2c1a99f986eb8aaea2290e2b1fece8b24de0807a31344363f Jul 06 19:24:52.300631 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.300614 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:24:52.302276 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:52.302251 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode817f8b8_1d4a_445d_91c4_6c6d77d2bd28.slice/crio-82687622f3a40ba7e1746b43fd55a13cb97961cfbc4a698dc92e34b3217809b6 WatchSource:0}: Error finding container 82687622f3a40ba7e1746b43fd55a13cb97961cfbc4a698dc92e34b3217809b6: Status 404 returned error can't find the container with id 82687622f3a40ba7e1746b43fd55a13cb97961cfbc4a698dc92e34b3217809b6 Jul 06 19:24:52.307080 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.307060 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:24:52.308714 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:52.308692 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbf09284_56b5_453a_8ed0_c33a09b04265.slice/crio-5d6e803f9c0e6e01241f81e1d5a4c5bbd01447273bc523e569f2d55cf1deac10 WatchSource:0}: Error finding container 5d6e803f9c0e6e01241f81e1d5a4c5bbd01447273bc523e569f2d55cf1deac10: Status 404 returned error can't find the container with id 5d6e803f9c0e6e01241f81e1d5a4c5bbd01447273bc523e569f2d55cf1deac10 Jul 06 19:24:52.312322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.312296 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ljx8t" Jul 06 19:24:52.316276 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:52.316245 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663a3e6c_13db_45ac_b7fd_0efd27a4b159.slice/crio-5078b45221d5d50b86ecf610ca38e2e7490c82e159f8d5725c4d37e342d3a952 WatchSource:0}: Error finding container 5078b45221d5d50b86ecf610ca38e2e7490c82e159f8d5725c4d37e342d3a952: Status 404 returned error can't find the container with id 5078b45221d5d50b86ecf610ca38e2e7490c82e159f8d5725c4d37e342d3a952 Jul 06 19:24:52.320531 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.320508 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z2r5s" Jul 06 19:24:52.321697 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:52.321675 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f9a6e3_08f5_4aa1_aba6_f07ff314e006.slice/crio-e924f278d56f63b03322d32c99f8253ae94bae581cc94d2eff90cbd3e8ea635a WatchSource:0}: Error finding container e924f278d56f63b03322d32c99f8253ae94bae581cc94d2eff90cbd3e8ea635a: Status 404 returned error can't find the container with id e924f278d56f63b03322d32c99f8253ae94bae581cc94d2eff90cbd3e8ea635a Jul 06 19:24:52.326497 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.326481 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:24:52.329040 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:24:52.329017 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09767752_33c4_4574_b852_5d40a6c8929f.slice/crio-c44fe1e1629698c9f0ab946f54791a83da881349431bd9ffe1ebade67f116895 WatchSource:0}: Error finding container c44fe1e1629698c9f0ab946f54791a83da881349431bd9ffe1ebade67f116895: Status 404 returned error can't find the container with id c44fe1e1629698c9f0ab946f54791a83da881349431bd9ffe1ebade67f116895 Jul 06 19:24:52.624496 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.624400 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:52.624649 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.624461 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:52.624649 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.624551 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:52.624649 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.624587 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:52.624649 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.624614 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs podName:ecb7b845-f496-49e7-87e3-a47752a5944b nodeName:}" failed. No retries permitted until 2026-07-06 19:24:53.624600105 +0000 UTC m=+4.129885124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs") pod "network-metrics-daemon-5942g" (UID: "ecb7b845-f496-49e7-87e3-a47752a5944b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:52.624649 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.624637 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret podName:b2cbb8d1-a657-43ca-a8d1-bc09a049a710 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:53.624623967 +0000 UTC m=+4.129908983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret") pod "global-pull-secret-syncer-cjr7k" (UID: "b2cbb8d1-a657-43ca-a8d1-bc09a049a710") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:52.725580 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.725539 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmz55\" (UniqueName: \"kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55\") pod \"network-check-target-ltflt\" (UID: \"8acd900e-9116-4692-a959-3ecf8035a515\") " pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:24:52.725756 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.725736 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:24:52.725756 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.725755 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:24:52.725872 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.725768 2572 projected.go:196] Error preparing data for projected volume kube-api-access-hmz55 for pod openshift-network-diagnostics/network-check-target-ltflt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:24:52.725872 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:52.725827 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55 podName:8acd900e-9116-4692-a959-3ecf8035a515 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:53.725809282 +0000 UTC m=+4.231094356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmz55" (UniqueName: "kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55") pod "network-check-target-ltflt" (UID: "8acd900e-9116-4692-a959-3ecf8035a515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:24:52.937543 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.937425 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:19:51 +0000 UTC" deadline="2028-03-27 19:33:49.571781864 +0000 UTC" Jul 06 19:24:52.937543 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:52.937518 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15120h8m56.634293163s" Jul 06 19:24:53.169924 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.169876 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:53.170713 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:53.170059 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:24:53.170713 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.170075 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:53.170713 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:53.170208 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:24:53.188744 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.188657 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z2r5s" event={"ID":"09767752-33c4-4574-b852-5d40a6c8929f","Type":"ContainerStarted","Data":"c44fe1e1629698c9f0ab946f54791a83da881349431bd9ffe1ebade67f116895"} Jul 06 19:24:53.195611 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.195429 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ljx8t" event={"ID":"68f9a6e3-08f5-4aa1-aba6-f07ff314e006","Type":"ContainerStarted","Data":"e924f278d56f63b03322d32c99f8253ae94bae581cc94d2eff90cbd3e8ea635a"} Jul 06 19:24:53.197983 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.197876 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7w9f7" event={"ID":"9621c68e-6ddf-4ff6-a451-062347974270","Type":"ContainerStarted","Data":"af31e71138b166d2c1a99f986eb8aaea2290e2b1fece8b24de0807a31344363f"} Jul 06 19:24:53.199835 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.199754 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dhd89" event={"ID":"8b19f7e0-96e8-45b3-8b56-eb3651786216","Type":"ContainerStarted","Data":"a9db1aa95365f613394331229c3d7364f7186cfb8422f6d69dfe105a27a76caf"} Jul 06 19:24:53.209784 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.209747 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" event={"ID":"8614ab98-8436-487f-8a3b-4485b54c043f","Type":"ContainerStarted","Data":"c0ba2e914c437cb0b13c223119e6d9159e618efe6c1ed1d825658b4fd5f81ba7"} Jul 06 19:24:53.228774 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.228672 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5pnv2" event={"ID":"663a3e6c-13db-45ac-b7fd-0efd27a4b159","Type":"ContainerStarted","Data":"5078b45221d5d50b86ecf610ca38e2e7490c82e159f8d5725c4d37e342d3a952"} Jul 06 19:24:53.251326 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.251216 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" event={"ID":"cbf09284-56b5-453a-8ed0-c33a09b04265","Type":"ContainerStarted","Data":"5d6e803f9c0e6e01241f81e1d5a4c5bbd01447273bc523e569f2d55cf1deac10"} Jul 06 19:24:53.264207 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.264147 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqg6b" event={"ID":"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28","Type":"ContainerStarted","Data":"82687622f3a40ba7e1746b43fd55a13cb97961cfbc4a698dc92e34b3217809b6"} Jul 06 19:24:53.271440 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.271402 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9crmz" event={"ID":"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd","Type":"ContainerStarted","Data":"45e7c85a5abfabf840641fd9ef2b910913370a9d3f781c3cac4aa4026c570734"} Jul 06 19:24:53.634537 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.633645 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:53.634537 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.633745 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:53.634537 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:53.633898 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:53.634537 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:53.633975 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs podName:ecb7b845-f496-49e7-87e3-a47752a5944b nodeName:}" failed. No retries permitted until 2026-07-06 19:24:55.633957893 +0000 UTC m=+6.139242910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs") pod "network-metrics-daemon-5942g" (UID: "ecb7b845-f496-49e7-87e3-a47752a5944b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:53.634537 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:53.634392 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:53.634537 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:53.634445 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret podName:b2cbb8d1-a657-43ca-a8d1-bc09a049a710 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:55.634429145 +0000 UTC m=+6.139714165 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret") pod "global-pull-secret-syncer-cjr7k" (UID: "b2cbb8d1-a657-43ca-a8d1-bc09a049a710") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:53.735044 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:53.735006 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmz55\" (UniqueName: \"kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55\") pod \"network-check-target-ltflt\" (UID: \"8acd900e-9116-4692-a959-3ecf8035a515\") " pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:24:53.735233 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:53.735210 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:24:53.735307 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:53.735244 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:24:53.735307 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:53.735258 2572 projected.go:196] Error preparing data for projected volume kube-api-access-hmz55 for pod openshift-network-diagnostics/network-check-target-ltflt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:24:53.735404 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:53.735328 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55 podName:8acd900e-9116-4692-a959-3ecf8035a515 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:55.735304935 +0000 UTC m=+6.240589963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmz55" (UniqueName: "kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55") pod "network-check-target-ltflt" (UID: "8acd900e-9116-4692-a959-3ecf8035a515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:24:54.170444 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:54.170412 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:24:54.170955 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:54.170597 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:24:55.170113 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:55.170020 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:55.170272 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:55.170158 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:24:55.170584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:55.170563 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:55.171069 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:55.170661 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:24:55.650279 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:55.650185 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:55.650279 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:55.650276 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:55.650494 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:55.650391 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:55.650494 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:55.650455 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs podName:ecb7b845-f496-49e7-87e3-a47752a5944b nodeName:}" failed. No retries permitted until 2026-07-06 19:24:59.650435704 +0000 UTC m=+10.155720733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs") pod "network-metrics-daemon-5942g" (UID: "ecb7b845-f496-49e7-87e3-a47752a5944b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:55.650920 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:55.650881 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:55.651049 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:55.650959 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret podName:b2cbb8d1-a657-43ca-a8d1-bc09a049a710 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:59.65094355 +0000 UTC m=+10.156228581 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret") pod "global-pull-secret-syncer-cjr7k" (UID: "b2cbb8d1-a657-43ca-a8d1-bc09a049a710") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:55.751944 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:55.751580 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmz55\" (UniqueName: \"kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55\") pod \"network-check-target-ltflt\" (UID: \"8acd900e-9116-4692-a959-3ecf8035a515\") " pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:24:55.751944 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:55.751764 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:24:55.751944 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:55.751786 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:24:55.751944 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:55.751798 2572 projected.go:196] Error preparing data for projected volume kube-api-access-hmz55 for pod openshift-network-diagnostics/network-check-target-ltflt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:24:55.751944 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:55.751858 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55 podName:8acd900e-9116-4692-a959-3ecf8035a515 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:59.751840799 +0000 UTC m=+10.257125880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmz55" (UniqueName: "kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55") pod "network-check-target-ltflt" (UID: "8acd900e-9116-4692-a959-3ecf8035a515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:24:56.169690 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:56.169651 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:24:56.169929 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:56.169885 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:24:57.169462 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:57.169423 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:57.169942 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:57.169581 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:24:57.170052 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:57.170025 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:57.170196 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:57.170122 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:24:58.171265 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:58.171206 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:24:58.171703 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:58.171345 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:24:59.169943 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:59.169895 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:59.170117 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:59.169916 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:59.170117 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:59.170064 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:24:59.170222 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:59.170179 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:24:59.683727 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:59.683687 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:24:59.684209 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:59.683760 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:24:59.684209 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:59.683901 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:59.684209 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:59.683978 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret podName:b2cbb8d1-a657-43ca-a8d1-bc09a049a710 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:07.683960404 +0000 UTC m=+18.189245438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret") pod "global-pull-secret-syncer-cjr7k" (UID: "b2cbb8d1-a657-43ca-a8d1-bc09a049a710") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:24:59.684209 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:59.683998 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:59.684209 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:59.684036 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs podName:ecb7b845-f496-49e7-87e3-a47752a5944b nodeName:}" failed. No retries permitted until 2026-07-06 19:25:07.684024747 +0000 UTC m=+18.189309778 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs") pod "network-metrics-daemon-5942g" (UID: "ecb7b845-f496-49e7-87e3-a47752a5944b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:24:59.784196 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:24:59.784154 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmz55\" (UniqueName: \"kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55\") pod \"network-check-target-ltflt\" (UID: \"8acd900e-9116-4692-a959-3ecf8035a515\") " pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:24:59.784390 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:59.784369 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:24:59.784390 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:59.784388 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:24:59.784504 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:59.784401 2572 projected.go:196] Error preparing data for projected volume kube-api-access-hmz55 for pod openshift-network-diagnostics/network-check-target-ltflt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:24:59.784504 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:24:59.784462 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55 podName:8acd900e-9116-4692-a959-3ecf8035a515 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:07.784443811 +0000 UTC m=+18.289728833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmz55" (UniqueName: "kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55") pod "network-check-target-ltflt" (UID: "8acd900e-9116-4692-a959-3ecf8035a515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:25:00.171024 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:00.170484 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:00.171024 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:00.170595 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:01.169447 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:01.169343 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:01.169961 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:01.169479 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:01.169961 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:01.169518 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:01.169961 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:01.169602 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:02.169698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:02.169660 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:02.170164 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:02.169796 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:03.169350 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:03.169316 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:03.169540 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:03.169316 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:03.169540 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:03.169441 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:03.169540 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:03.169507 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:04.170989 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:04.169789 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:04.170989 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:04.169950 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:05.169606 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:05.169564 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:05.169789 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:05.169718 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:05.169838 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:05.169795 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:05.169977 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:05.169924 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:06.169460 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:06.169219 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:06.169921 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:06.169566 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:07.169714 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:07.169675 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:07.169714 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:07.169704 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:07.170297 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:07.169835 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:07.170297 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:07.169960 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:07.744260 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:07.744061 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:07.744260 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:07.744122 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:07.744260 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:07.744163 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:25:07.744260 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:07.744222 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:25:07.744260 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:07.744233 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret podName:b2cbb8d1-a657-43ca-a8d1-bc09a049a710 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:23.744211997 +0000 UTC m=+34.249497034 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret") pod "global-pull-secret-syncer-cjr7k" (UID: "b2cbb8d1-a657-43ca-a8d1-bc09a049a710") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:25:07.744459 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:07.744267 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs podName:ecb7b845-f496-49e7-87e3-a47752a5944b nodeName:}" failed. No retries permitted until 2026-07-06 19:25:23.744254847 +0000 UTC m=+34.249539863 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs") pod "network-metrics-daemon-5942g" (UID: "ecb7b845-f496-49e7-87e3-a47752a5944b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:25:07.845497 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:07.845464 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmz55\" (UniqueName: \"kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55\") pod \"network-check-target-ltflt\" (UID: \"8acd900e-9116-4692-a959-3ecf8035a515\") " pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:07.845693 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:07.845651 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:25:07.845693 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:07.845676 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:25:07.845693 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:07.845688 2572 projected.go:196] Error preparing data for projected volume kube-api-access-hmz55 for pod openshift-network-diagnostics/network-check-target-ltflt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:25:07.845853 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:07.845759 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55 podName:8acd900e-9116-4692-a959-3ecf8035a515 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:23.845739482 +0000 UTC m=+34.351024500 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmz55" (UniqueName: "kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55") pod "network-check-target-ltflt" (UID: "8acd900e-9116-4692-a959-3ecf8035a515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:25:08.169607 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:08.169513 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:08.169785 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:08.169653 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:09.170063 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:09.170025 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:09.170533 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:09.170082 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:09.170533 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:09.170149 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:09.170533 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:09.170307 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:10.170276 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:10.170243 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:10.170656 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:10.170362 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:11.169163 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:11.169126 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:11.169369 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:11.169250 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:11.169369 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:11.169316 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:11.169472 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:11.169431 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:12.173681 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.173392 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:12.174098 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:12.173679 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:12.322806 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.322744 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-35.ec2.internal" event={"ID":"a91a71edc512a42ccb4fd1df84711f15","Type":"ContainerStarted","Data":"65a5a523b9a3a79ecb1bf55e8354588c389c761802e24102e7491f86c75a782f"} Jul 06 19:25:12.324832 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.324811 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:25:12.325284 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.325256 2572 generic.go:356] "Generic (PLEG): container finished" podID="cbf09284-56b5-453a-8ed0-c33a09b04265" containerID="b72969773d16b72f2cc58acdaa6a390660982cbf625d1504522b4ce271db41df" exitCode=1 Jul 06 19:25:12.325402 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.325295 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" event={"ID":"cbf09284-56b5-453a-8ed0-c33a09b04265","Type":"ContainerStarted","Data":"9d42f8cf913ece3469d2db94361a7f8d8438e7670a46c2448e6c3d7c68e43348"} Jul 06 19:25:12.325402 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.325393 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" event={"ID":"cbf09284-56b5-453a-8ed0-c33a09b04265","Type":"ContainerDied","Data":"b72969773d16b72f2cc58acdaa6a390660982cbf625d1504522b4ce271db41df"} Jul 06 19:25:12.325517 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.325447 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" event={"ID":"cbf09284-56b5-453a-8ed0-c33a09b04265","Type":"ContainerStarted","Data":"9b262f4501f54644f47c00ae7d044b64d1a56c70c69cc4a97245da831e69db51"} Jul 06 19:25:12.326799 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.326770 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9crmz" event={"ID":"fcffc633-f4f2-4d7f-8fd2-d4c1819655dd","Type":"ContainerStarted","Data":"ffa2ffae2b65ec4f128c5ee0aa942ae030f0cb6533563a5d7eb3743dd1cff406"} Jul 06 19:25:12.328216 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.328171 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dhd89" event={"ID":"8b19f7e0-96e8-45b3-8b56-eb3651786216","Type":"ContainerStarted","Data":"d795c316b2755b2293ffcedfbc4ed13b0de02359d19bbbff22a36ea25c3a16a8"} Jul 06 19:25:12.344524 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.344473 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-35.ec2.internal" podStartSLOduration=21.344457284 podStartE2EDuration="21.344457284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:24:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:12.344047321 +0000 UTC m=+22.849332362" watchObservedRunningTime="2026-07-06 19:25:12.344457284 +0000 UTC m=+22.849742323" Jul 06 19:25:12.374667 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.374429 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9crmz" podStartSLOduration=2.6152204770000003 podStartE2EDuration="22.374407387s" totalImagesPullingTime="19.75918691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:12.373373221 +0000 UTC m=+22.878658260" watchObservedRunningTime="2026-07-06 19:25:12.374407387 +0000 UTC m=+22.879692426" Jul 06 19:25:12.400336 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:12.400287 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dhd89" podStartSLOduration=2.76225364 podStartE2EDuration="22.400269809s" totalImagesPullingTime="19.638016169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:12.399712415 +0000 UTC m=+22.904997454" watchObservedRunningTime="2026-07-06 19:25:12.400269809 +0000 UTC m=+22.905554847" Jul 06 19:25:13.169199 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.169011 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:13.169354 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.169071 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:13.169354 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:13.169280 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:13.169354 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:13.169342 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:13.332728 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.332643 2572 generic.go:356] "Generic (PLEG): container finished" podID="e817f8b8-1d4a-445d-91c4-6c6d77d2bd28" containerID="9e34f619ab0edf510ec73c01acf4c4aa7e11c3a9ca61a0ca1f552d1e5b18589d" exitCode=0 Jul 06 19:25:13.333761 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.332754 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqg6b" event={"ID":"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28","Type":"ContainerDied","Data":"9e34f619ab0edf510ec73c01acf4c4aa7e11c3a9ca61a0ca1f552d1e5b18589d"} Jul 06 19:25:13.334543 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.334507 2572 generic.go:356] "Generic (PLEG): container finished" podID="5dcde92490703d276672775cee39ea5c" containerID="210051c63435686540646a9eee94c76d65a09b19fc82bc271d744d8e5cf2f60b" exitCode=0 Jul 06 19:25:13.334607 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.334591 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" event={"ID":"5dcde92490703d276672775cee39ea5c","Type":"ContainerDied","Data":"210051c63435686540646a9eee94c76d65a09b19fc82bc271d744d8e5cf2f60b"} Jul 06 19:25:13.336564 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.336535 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z2r5s" event={"ID":"09767752-33c4-4574-b852-5d40a6c8929f","Type":"ContainerStarted","Data":"f583166135d72aaa0244e3b1e6a3bac90c445ddc47eb91db2ab41fb070bc5830"} Jul 06 19:25:13.338732 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.338706 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ljx8t" event={"ID":"68f9a6e3-08f5-4aa1-aba6-f07ff314e006","Type":"ContainerStarted","Data":"5e34f0527ea76a14d65e971c19b92a0f45f373f615e6d891c9f7d6a07c7eb52a"} Jul 06 19:25:13.340349 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.340323 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7w9f7" event={"ID":"9621c68e-6ddf-4ff6-a451-062347974270","Type":"ContainerStarted","Data":"79881651cc4a0a399aa7f1cffee757d7355d729f676625d7f99dcc418ae65382"} Jul 06 19:25:13.342096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.342071 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" event={"ID":"8614ab98-8436-487f-8a3b-4485b54c043f","Type":"ContainerStarted","Data":"5e9f33b61365a1022bd174dc626e23fca7cd38a7a1b6cb80336b41c71a9a423e"} Jul 06 19:25:13.343713 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.343688 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5pnv2" event={"ID":"663a3e6c-13db-45ac-b7fd-0efd27a4b159","Type":"ContainerStarted","Data":"054bf7d31ebff3ab01cdedca43d4aeaabe2939dc0dd2a7f0d5b7baf4d8dc50db"} Jul 06 19:25:13.346994 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.346975 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:25:13.347443 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.347417 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" event={"ID":"cbf09284-56b5-453a-8ed0-c33a09b04265","Type":"ContainerStarted","Data":"4b33ef18191ffd8006ed18dfb367795e3025401960f46fca5c137b81c7c29f10"} Jul 06 19:25:13.347558 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.347504 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" event={"ID":"cbf09284-56b5-453a-8ed0-c33a09b04265","Type":"ContainerStarted","Data":"6c7a308b8382a9b5e924b0025ae7c46a048b3fcc03f9da2c30cf71b3e4e5593b"} Jul 06 19:25:13.347558 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.347554 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" event={"ID":"cbf09284-56b5-453a-8ed0-c33a09b04265","Type":"ContainerStarted","Data":"b5b2fadeceb4ff676066fdc796b3b42ee1d98409000c09ddef449b202fdc3d59"} Jul 06 19:25:13.392590 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.392479 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z2r5s" podStartSLOduration=3.77064529 podStartE2EDuration="23.392456247s" totalImagesPullingTime="19.621810957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:13.37748496 +0000 UTC m=+23.882769997" watchObservedRunningTime="2026-07-06 19:25:13.392456247 +0000 UTC m=+23.897741485" Jul 06 19:25:13.406261 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.406202 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5pnv2" podStartSLOduration=3.773584905 podStartE2EDuration="23.406178736s" totalImagesPullingTime="19.632593831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:13.406106228 +0000 UTC m=+23.911391268" watchObservedRunningTime="2026-07-06 19:25:13.406178736 +0000 UTC m=+23.911463778" Jul 06 19:25:13.425266 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.425220 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7w9f7" podStartSLOduration=3.769459318 podStartE2EDuration="23.425208781s" totalImagesPullingTime="19.655749463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:13.425099864 +0000 UTC m=+23.930384903" watchObservedRunningTime="2026-07-06 19:25:13.425208781 +0000 UTC m=+23.930493820" Jul 06 19:25:13.440145 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:13.440098 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ljx8t" podStartSLOduration=3.807720786 podStartE2EDuration="23.440082295s" totalImagesPullingTime="19.632361509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:13.439651144 +0000 UTC m=+23.944936183" watchObservedRunningTime="2026-07-06 19:25:13.440082295 +0000 UTC m=+23.945367332" Jul 06 19:25:14.169532 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:14.169513 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:14.169680 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:14.169622 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:14.176173 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:14.176145 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 19:25:14.353137 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:14.353036 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" event={"ID":"8614ab98-8436-487f-8a3b-4485b54c043f","Type":"ContainerStarted","Data":"a2630836e8feead77901bef31272e7996ffe1de6bc4b6688ba966f001703b7b1"} Jul 06 19:25:14.354671 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:14.354638 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" event={"ID":"5dcde92490703d276672775cee39ea5c","Type":"ContainerStarted","Data":"518755d929dc1bc67f7d8d6d6c3f88b36c61f9f70d0c24e2c033b1473e51797d"} Jul 06 19:25:14.970465 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:14.970294 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T19:25:14.176165998Z","UUID":"c4cf8449-9712-47d2-87a2-fa46b1b60255","Handler":null,"Name":"","Endpoint":""} Jul 06 19:25:14.971817 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:14.971796 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 19:25:14.971957 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:14.971834 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 19:25:15.170044 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:15.170006 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:15.170229 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:15.170006 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:15.170229 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:15.170145 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:15.170350 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:15.170262 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:15.360384 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:15.360274 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:25:15.360722 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:15.360700 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" event={"ID":"cbf09284-56b5-453a-8ed0-c33a09b04265","Type":"ContainerStarted","Data":"2ed8b83b666dd1f21e95940408a988faec6c93b7f2e4abc7f843e8e88db01001"} Jul 06 19:25:16.173295 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:16.173269 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:16.173491 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:16.173387 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:16.364888 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:16.364852 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" event={"ID":"8614ab98-8436-487f-8a3b-4485b54c043f","Type":"ContainerStarted","Data":"757eb4283f3864298520feb638dc0778fe7d6b736ff75972e6bcea437dfd9d80"} Jul 06 19:25:16.384888 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:16.384835 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-35.ec2.internal" podStartSLOduration=25.384816611 podStartE2EDuration="25.384816611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:24:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:14.396626535 +0000 UTC m=+24.901911573" watchObservedRunningTime="2026-07-06 19:25:16.384816611 +0000 UTC m=+26.890101649" Jul 06 19:25:16.385206 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:16.385185 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zvt8d" podStartSLOduration=4.222568 podStartE2EDuration="26.385176813s" totalImagesPullingTime="22.162608813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:16.38491621 +0000 UTC m=+26.890201240" watchObservedRunningTime="2026-07-06 19:25:16.385176813 +0000 UTC m=+26.890461855" Jul 06 19:25:16.464461 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:16.464369 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:25:16.465051 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:16.465032 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:25:17.169887 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:17.169638 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:17.169887 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:17.169691 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:17.170095 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:17.169957 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:17.170095 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:17.169992 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:17.370520 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:17.370494 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:25:17.371099 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:17.370853 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" event={"ID":"cbf09284-56b5-453a-8ed0-c33a09b04265","Type":"ContainerStarted","Data":"59bb89b7d516043fe50564ea7ee1d8a684d2d556a511f235a48293d5680d37dd"} Jul 06 19:25:17.371181 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:17.371099 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:25:17.371332 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:17.371309 2572 scope.go:117] "RemoveContainer" containerID="b72969773d16b72f2cc58acdaa6a390660982cbf625d1504522b4ce271db41df" Jul 06 19:25:17.371618 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:17.371603 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5pnv2" Jul 06 19:25:18.175531 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.175490 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:18.175695 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:18.175611 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:18.378211 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.378178 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:25:18.379360 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.379331 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" event={"ID":"cbf09284-56b5-453a-8ed0-c33a09b04265","Type":"ContainerStarted","Data":"8f69b13a993053ca39fc6935f9e82e5ab2ae7f6e8bb6dcb588ef36ccd1f45a22"} Jul 06 19:25:18.379770 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.379745 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:25:18.379900 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.379823 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:25:18.379900 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.379871 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:25:18.398287 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.398259 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:25:18.398429 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.398403 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:25:18.409811 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.409751 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" podStartSLOduration=8.691564575 podStartE2EDuration="28.409734903s" totalImagesPullingTime="19.718170328s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:18.409379722 +0000 UTC m=+28.914664760" watchObservedRunningTime="2026-07-06 19:25:18.409734903 +0000 UTC m=+28.915019941" Jul 06 19:25:18.831260 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.831180 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ltflt"] Jul 06 19:25:18.831418 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.831330 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:18.831685 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:18.831414 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:18.833231 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.833204 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cjr7k"] Jul 06 19:25:18.833377 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.833326 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:18.833451 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:18.833422 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:18.834044 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.834019 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5942g"] Jul 06 19:25:18.834553 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:18.834123 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:18.834553 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:18.834217 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:20.170314 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:20.170276 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:20.170842 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:20.170413 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:20.170842 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:20.170436 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:20.170842 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:20.170552 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:21.169789 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:21.169746 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:21.169988 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:21.169862 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:21.388037 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:21.387956 2572 generic.go:356] "Generic (PLEG): container finished" podID="e817f8b8-1d4a-445d-91c4-6c6d77d2bd28" containerID="60a99d64e9668978267ad5217c4ed23b714aa0e11e34a3a77845aaf8d6ae47d4" exitCode=0 Jul 06 19:25:21.388037 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:21.388014 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqg6b" event={"ID":"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28","Type":"ContainerDied","Data":"60a99d64e9668978267ad5217c4ed23b714aa0e11e34a3a77845aaf8d6ae47d4"} Jul 06 19:25:22.169994 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:22.169958 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:22.170187 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:22.169972 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:22.170187 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:22.170097 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:22.170187 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:22.170140 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:22.392250 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:22.392214 2572 generic.go:356] "Generic (PLEG): container finished" podID="e817f8b8-1d4a-445d-91c4-6c6d77d2bd28" containerID="83c7e540d391fdd17c48ef0f046a278d576670095fdc815cfb4eaa17435de1e0" exitCode=0 Jul 06 19:25:22.392596 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:22.392260 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqg6b" event={"ID":"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28","Type":"ContainerDied","Data":"83c7e540d391fdd17c48ef0f046a278d576670095fdc815cfb4eaa17435de1e0"} Jul 06 19:25:23.169441 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:23.169414 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:23.169576 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:23.169537 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltflt" podUID="8acd900e-9116-4692-a959-3ecf8035a515" Jul 06 19:25:23.396724 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:23.396633 2572 generic.go:356] "Generic (PLEG): container finished" podID="e817f8b8-1d4a-445d-91c4-6c6d77d2bd28" containerID="a5585f8ab3e15dd1b32989a7a7b3c677ac71176916bc85caa88b1ab4d24c8e27" exitCode=0 Jul 06 19:25:23.396724 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:23.396691 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqg6b" event={"ID":"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28","Type":"ContainerDied","Data":"a5585f8ab3e15dd1b32989a7a7b3c677ac71176916bc85caa88b1ab4d24c8e27"} Jul 06 19:25:23.767700 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:23.767610 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:23.767700 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:23.767659 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:23.767866 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:23.767765 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:25:23.767866 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:23.767767 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:25:23.767866 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:23.767821 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret podName:b2cbb8d1-a657-43ca-a8d1-bc09a049a710 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:55.767807092 +0000 UTC m=+66.273092121 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret") pod "global-pull-secret-syncer-cjr7k" (UID: "b2cbb8d1-a657-43ca-a8d1-bc09a049a710") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:25:23.767866 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:23.767838 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs podName:ecb7b845-f496-49e7-87e3-a47752a5944b nodeName:}" failed. No retries permitted until 2026-07-06 19:25:55.767831461 +0000 UTC m=+66.273116477 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs") pod "network-metrics-daemon-5942g" (UID: "ecb7b845-f496-49e7-87e3-a47752a5944b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:25:23.868810 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:23.868776 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmz55\" (UniqueName: \"kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55\") pod \"network-check-target-ltflt\" (UID: \"8acd900e-9116-4692-a959-3ecf8035a515\") " pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:23.868973 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:23.868927 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:25:23.868973 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:23.868942 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:25:23.868973 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:23.868951 2572 projected.go:196] Error preparing data for projected volume kube-api-access-hmz55 for pod openshift-network-diagnostics/network-check-target-ltflt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:25:23.869093 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:23.868996 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55 podName:8acd900e-9116-4692-a959-3ecf8035a515 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:55.868984447 +0000 UTC m=+66.374269462 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmz55" (UniqueName: "kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55") pod "network-check-target-ltflt" (UID: "8acd900e-9116-4692-a959-3ecf8035a515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:25:24.170176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.170089 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:24.170176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.170121 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:24.170401 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:24.170225 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjr7k" podUID="b2cbb8d1-a657-43ca-a8d1-bc09a049a710" Jul 06 19:25:24.170401 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:24.170376 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5942g" podUID="ecb7b845-f496-49e7-87e3-a47752a5944b" Jul 06 19:25:24.772327 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.772247 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-35.ec2.internal" event="NodeReady" Jul 06 19:25:24.772855 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.772409 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 19:25:24.814302 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.814258 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74b85d8d69-gq2gp"] Jul 06 19:25:24.816985 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.816956 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:24.820563 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.820501 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 19:25:24.820981 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.820799 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 19:25:24.820981 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.820817 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-plcs2\"" Jul 06 19:25:24.821281 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.821261 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 19:25:24.826855 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.826834 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-k6bbz"] Jul 06 19:25:24.830654 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.830631 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx"] Jul 06 19:25:24.830846 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.830819 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-k6bbz" Jul 06 19:25:24.833260 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.833228 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 19:25:24.833643 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.833598 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-rg9vj"] Jul 06 19:25:24.833822 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.833801 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" Jul 06 19:25:24.834361 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.833994 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:25:24.834361 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.834126 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 19:25:24.834493 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.834473 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-d62cl\"" Jul 06 19:25:24.836187 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.836167 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 06 19:25:24.836321 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.836193 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 06 19:25:24.836389 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.836327 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s"] Jul 06 19:25:24.836483 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.836463 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:24.836711 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.836689 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:25:24.837082 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.837055 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 06 19:25:24.837082 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.837066 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-s2bpd\"" Jul 06 19:25:24.838986 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.838968 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-6cqbs"] Jul 06 19:25:24.839290 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.839269 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:24.839716 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.839701 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-wlrkz\"" Jul 06 19:25:24.839847 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.839835 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 19:25:24.839971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.839957 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 19:25:24.840051 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.839702 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 19:25:24.840126 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.840112 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 19:25:24.841822 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.841804 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s"] Jul 06 19:25:24.841946 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.841932 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:24.845118 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.844082 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:25:24.845118 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.844539 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 19:25:24.845707 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.845684 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:25:24.846094 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.846077 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-s7t8b\"" Jul 06 19:25:24.846467 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.846441 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 19:25:24.847988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.847013 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 19:25:24.847988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.847265 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b85d8d69-gq2gp"] Jul 06 19:25:24.847988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.847399 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" Jul 06 19:25:24.847988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.844250 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 19:25:24.851628 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.851608 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-xpp7z\"" Jul 06 19:25:24.851960 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.851647 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 19:25:24.854717 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.854697 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 19:25:24.855023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.854999 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx"] Jul 06 19:25:24.855941 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.855892 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 19:25:24.856270 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.856249 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 19:25:24.856459 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.856412 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:25:24.856554 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.856474 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 19:25:24.856554 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.856491 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5db6z\"" Jul 06 19:25:24.857115 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.857099 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-k6bbz"] Jul 06 19:25:24.857568 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.857551 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 19:25:24.858019 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.858000 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-rg9vj"] Jul 06 19:25:24.859124 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.858775 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s"] Jul 06 19:25:24.859762 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.859723 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-6cqbs"] Jul 06 19:25:24.860612 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.860596 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gd48l"] Jul 06 19:25:24.864471 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.864430 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hfr9w"] Jul 06 19:25:24.864607 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.864542 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:24.867788 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.867770 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-756fc79d99-fvh2j"] Jul 06 19:25:24.867937 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.867920 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hfr9w" Jul 06 19:25:24.868031 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.868012 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 19:25:24.868088 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.868038 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-khkrm\"" Jul 06 19:25:24.868685 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.868664 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 19:25:24.870832 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.870815 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 19:25:24.870949 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.870881 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 19:25:24.871131 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.871112 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 19:25:24.871241 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.871212 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vrv2c\"" Jul 06 19:25:24.872372 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.872351 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd"] Jul 06 19:25:24.872951 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.872931 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:24.875769 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.875745 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s"] Jul 06 19:25:24.875877 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.875791 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hfr9w"] Jul 06 19:25:24.875955 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.875878 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:24.878449 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.877318 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 19:25:24.878449 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.877357 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zp7lr\"" Jul 06 19:25:24.878449 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.877421 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 19:25:24.878449 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.877559 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 19:25:24.878449 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.877629 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 19:25:24.878449 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.877864 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 19:25:24.878449 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.878445 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 19:25:24.879088 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.878599 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gd48l"] Jul 06 19:25:24.879464 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.879445 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 19:25:24.879718 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.879700 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd"] Jul 06 19:25:24.879950 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.879929 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 19:25:24.880059 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.879989 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 19:25:24.880059 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.879938 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2ltbj\"" Jul 06 19:25:24.880232 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.880198 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 19:25:24.896548 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.896520 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-756fc79d99-fvh2j"] Jul 06 19:25:24.976886 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.976853 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/940db99f-f328-4608-b40e-31243191d438-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-pkf7s\" (UID: \"940db99f-f328-4608-b40e-31243191d438\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" Jul 06 19:25:24.976886 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.976890 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-config\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:24.977122 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.976927 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbc5g\" (UniqueName: \"kubernetes.io/projected/2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112-kube-api-access-pbc5g\") pod \"kube-storage-version-migrator-operator-846bf668f7-6qvnx\" (UID: \"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" Jul 06 19:25:24.977122 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.976947 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/239c95d1-5bdb-4078-ad6b-9eb593957bb6-serving-cert\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:24.977122 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977020 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/239c95d1-5bdb-4078-ad6b-9eb593957bb6-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:24.977122 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977076 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-6qvnx\" (UID: \"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" Jul 06 19:25:24.977297 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977125 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:24.977297 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977154 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:24.977297 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977195 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/940db99f-f328-4608-b40e-31243191d438-config\") pod \"service-ca-operator-5d46d9ffd4-pkf7s\" (UID: \"940db99f-f328-4608-b40e-31243191d438\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" Jul 06 19:25:24.977297 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977221 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-6qvnx\" (UID: \"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" Jul 06 19:25:24.977297 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977238 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqf6l\" (UniqueName: \"kubernetes.io/projected/752bf161-7c8b-4af8-9d5d-3a7f9260e693-kube-api-access-mqf6l\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:24.977509 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977309 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:24.977509 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977338 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-bound-sa-token\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:24.977509 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977357 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-default-certificate\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:24.977509 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977403 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:24.977509 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977440 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/55445c8d-30b3-47cd-9479-1ca223c726e5-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:24.977509 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977471 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:24.977804 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977527 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-installation-pull-secrets\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:24.977804 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977552 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4eee88bf-1be6-492f-82eb-b64464b0199e-cert\") pod \"ingress-canary-hfr9w\" (UID: \"4eee88bf-1be6-492f-82eb-b64464b0199e\") " pod="openshift-ingress-canary/ingress-canary-hfr9w" Jul 06 19:25:24.977804 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977575 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plzrv\" (UniqueName: \"kubernetes.io/projected/239c95d1-5bdb-4078-ad6b-9eb593957bb6-kube-api-access-plzrv\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:24.977804 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977601 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-trusted-ca\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:24.977804 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977640 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6b2q\" (UniqueName: \"kubernetes.io/projected/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-kube-api-access-t6b2q\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:24.977804 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977666 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc9k4\" (UniqueName: \"kubernetes.io/projected/4eee88bf-1be6-492f-82eb-b64464b0199e-kube-api-access-jc9k4\") pod \"ingress-canary-hfr9w\" (UID: \"4eee88bf-1be6-492f-82eb-b64464b0199e\") " pod="openshift-ingress-canary/ingress-canary-hfr9w" Jul 06 19:25:24.977804 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977693 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/752bf161-7c8b-4af8-9d5d-3a7f9260e693-config-volume\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:24.977804 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977715 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:24.978179 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977810 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-image-registry-private-configuration\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:24.978179 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977854 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-certificates\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:24.978179 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977882 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/239c95d1-5bdb-4078-ad6b-9eb593957bb6-tmp\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:24.978179 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.977919 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/239c95d1-5bdb-4078-ad6b-9eb593957bb6-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:24.978179 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978032 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf9pr\" (UniqueName: \"kubernetes.io/projected/940db99f-f328-4608-b40e-31243191d438-kube-api-access-tf9pr\") pod \"service-ca-operator-5d46d9ffd4-pkf7s\" (UID: \"940db99f-f328-4608-b40e-31243191d438\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" Jul 06 19:25:24.978179 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978061 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-serving-cert\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:24.978179 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978085 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7cr7\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-kube-api-access-j7cr7\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:24.978179 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978111 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-trusted-ca\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:24.978179 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978146 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvjtg\" (UniqueName: \"kubernetes.io/projected/55445c8d-30b3-47cd-9479-1ca223c726e5-kube-api-access-gvjtg\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:24.978553 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978190 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-ca-trust-extracted\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:24.978553 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978217 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs44x\" (UniqueName: \"kubernetes.io/projected/7f1c75a3-c731-4050-82ba-444e5f4d5a9a-kube-api-access-vs44x\") pod \"volume-data-source-validator-5d456fdbcd-k6bbz\" (UID: \"7f1c75a3-c731-4050-82ba-444e5f4d5a9a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-k6bbz" Jul 06 19:25:24.978553 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978254 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/239c95d1-5bdb-4078-ad6b-9eb593957bb6-snapshots\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:24.978553 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978300 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcdzg\" (UniqueName: \"kubernetes.io/projected/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-kube-api-access-dcdzg\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:24.978553 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978333 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-stats-auth\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:24.978553 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978356 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jrjd\" (UniqueName: \"kubernetes.io/projected/2369af8d-444e-488a-97c3-692ee5138a01-kube-api-access-6jrjd\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:24.978553 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:24.978387 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/752bf161-7c8b-4af8-9d5d-3a7f9260e693-tmp-dir\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:25.054998 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.054711 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:25.079633 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079602 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-ca-trust-extracted\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.079633 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079637 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs44x\" (UniqueName: \"kubernetes.io/projected/7f1c75a3-c731-4050-82ba-444e5f4d5a9a-kube-api-access-vs44x\") pod \"volume-data-source-validator-5d456fdbcd-k6bbz\" (UID: \"7f1c75a3-c731-4050-82ba-444e5f4d5a9a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-k6bbz" Jul 06 19:25:25.079888 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079667 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/239c95d1-5bdb-4078-ad6b-9eb593957bb6-snapshots\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.079888 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079735 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcdzg\" (UniqueName: \"kubernetes.io/projected/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-kube-api-access-dcdzg\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:25.079888 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079763 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-stats-auth\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:25.079888 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079787 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jrjd\" (UniqueName: \"kubernetes.io/projected/2369af8d-444e-488a-97c3-692ee5138a01-kube-api-access-6jrjd\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:25.079888 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079813 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/752bf161-7c8b-4af8-9d5d-3a7f9260e693-tmp-dir\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:25.079888 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079844 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/940db99f-f328-4608-b40e-31243191d438-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-pkf7s\" (UID: \"940db99f-f328-4608-b40e-31243191d438\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" Jul 06 19:25:25.079888 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079868 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-config\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079895 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbc5g\" (UniqueName: \"kubernetes.io/projected/2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112-kube-api-access-pbc5g\") pod \"kube-storage-version-migrator-operator-846bf668f7-6qvnx\" (UID: \"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079935 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/239c95d1-5bdb-4078-ad6b-9eb593957bb6-serving-cert\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079960 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/239c95d1-5bdb-4078-ad6b-9eb593957bb6-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.079988 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-6qvnx\" (UID: \"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080022 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080048 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080068 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-ca-trust-extracted\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080073 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/940db99f-f328-4608-b40e-31243191d438-config\") pod \"service-ca-operator-5d46d9ffd4-pkf7s\" (UID: \"940db99f-f328-4608-b40e-31243191d438\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080117 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-6qvnx\" (UID: \"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080136 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqf6l\" (UniqueName: \"kubernetes.io/projected/752bf161-7c8b-4af8-9d5d-3a7f9260e693-kube-api-access-mqf6l\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080158 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:25.080223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080182 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-bound-sa-token\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.080740 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080508 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/239c95d1-5bdb-4078-ad6b-9eb593957bb6-snapshots\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.080740 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.080594 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:25.080740 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080626 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/752bf161-7c8b-4af8-9d5d-3a7f9260e693-tmp-dir\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:25.080740 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080628 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/940db99f-f328-4608-b40e-31243191d438-config\") pod \"service-ca-operator-5d46d9ffd4-pkf7s\" (UID: \"940db99f-f328-4608-b40e-31243191d438\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" Jul 06 19:25:25.080740 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.080654 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls podName:55445c8d-30b3-47cd-9479-1ca223c726e5 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:25.580634699 +0000 UTC m=+36.085919730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lx4sd" (UID: "55445c8d-30b3-47cd-9479-1ca223c726e5") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:25.080740 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.080695 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:25:25.081051 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.080759 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls podName:1e15d9b3-599b-4f3b-8571-12b7a36a6d3e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:25.580743127 +0000 UTC m=+36.086028160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-nmz4s" (UID: "1e15d9b3-599b-4f3b-8571-12b7a36a6d3e") : secret "samples-operator-tls" not found Jul 06 19:25:25.081051 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.080842 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:25.580830616 +0000 UTC m=+36.086115645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:25.081051 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080694 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-default-certificate\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:25.081051 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080876 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:25.081051 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080923 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/55445c8d-30b3-47cd-9479-1ca223c726e5-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:25.081051 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080955 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.081051 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.080986 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-installation-pull-secrets\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.081051 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081013 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4eee88bf-1be6-492f-82eb-b64464b0199e-cert\") pod \"ingress-canary-hfr9w\" (UID: \"4eee88bf-1be6-492f-82eb-b64464b0199e\") " pod="openshift-ingress-canary/ingress-canary-hfr9w" Jul 06 19:25:25.081051 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081040 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plzrv\" (UniqueName: \"kubernetes.io/projected/239c95d1-5bdb-4078-ad6b-9eb593957bb6-kube-api-access-plzrv\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.081497 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081058 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/239c95d1-5bdb-4078-ad6b-9eb593957bb6-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.081497 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.081141 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:25.081497 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.081232 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:25.581216782 +0000 UTC m=+36.086501806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : secret "router-metrics-certs-default" not found Jul 06 19:25:25.081651 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081579 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-config\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:25.081698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081665 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-6qvnx\" (UID: \"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" Jul 06 19:25:25.081698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081066 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-trusted-ca\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:25.081805 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081721 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6b2q\" (UniqueName: \"kubernetes.io/projected/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-kube-api-access-t6b2q\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:25.081805 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081750 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc9k4\" (UniqueName: \"kubernetes.io/projected/4eee88bf-1be6-492f-82eb-b64464b0199e-kube-api-access-jc9k4\") pod \"ingress-canary-hfr9w\" (UID: \"4eee88bf-1be6-492f-82eb-b64464b0199e\") " pod="openshift-ingress-canary/ingress-canary-hfr9w" Jul 06 19:25:25.081805 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081781 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/752bf161-7c8b-4af8-9d5d-3a7f9260e693-config-volume\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:25.081805 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081802 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:25.082029 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081856 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-image-registry-private-configuration\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.082029 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081895 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-certificates\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.082029 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081938 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/239c95d1-5bdb-4078-ad6b-9eb593957bb6-tmp\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.082029 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081961 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/239c95d1-5bdb-4078-ad6b-9eb593957bb6-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.082029 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.081992 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-trusted-ca\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:25.082029 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.082004 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf9pr\" (UniqueName: \"kubernetes.io/projected/940db99f-f328-4608-b40e-31243191d438-kube-api-access-tf9pr\") pod \"service-ca-operator-5d46d9ffd4-pkf7s\" (UID: \"940db99f-f328-4608-b40e-31243191d438\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" Jul 06 19:25:25.082318 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.082043 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-serving-cert\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:25.082318 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.082074 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7cr7\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-kube-api-access-j7cr7\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.082318 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.082110 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-trusted-ca\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.082318 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.082136 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvjtg\" (UniqueName: \"kubernetes.io/projected/55445c8d-30b3-47cd-9479-1ca223c726e5-kube-api-access-gvjtg\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:25.083064 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.083037 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/752bf161-7c8b-4af8-9d5d-3a7f9260e693-config-volume\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:25.083184 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.083130 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:25:25.083184 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.083173 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls podName:752bf161-7c8b-4af8-9d5d-3a7f9260e693 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:25.583158514 +0000 UTC m=+36.088443535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls") pod "dns-default-gd48l" (UID: "752bf161-7c8b-4af8-9d5d-3a7f9260e693") : secret "dns-default-metrics-tls" not found Jul 06 19:25:25.084235 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.084202 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/239c95d1-5bdb-4078-ad6b-9eb593957bb6-tmp\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.084788 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.084763 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-certificates\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.084922 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.084888 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:25:25.084986 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.084923 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74b85d8d69-gq2gp: secret "image-registry-tls" not found Jul 06 19:25:25.085047 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.084986 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls podName:cd7bc946-cbb8-4966-b8c5-6edb16cc366e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:25.584962987 +0000 UTC m=+36.090248022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls") pod "image-registry-74b85d8d69-gq2gp" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e") : secret "image-registry-tls" not found Jul 06 19:25:25.085214 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.085177 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-6qvnx\" (UID: \"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" Jul 06 19:25:25.085325 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.085251 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/55445c8d-30b3-47cd-9479-1ca223c726e5-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:25.085325 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.085257 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-installation-pull-secrets\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.086040 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.086015 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/239c95d1-5bdb-4078-ad6b-9eb593957bb6-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.086142 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.086055 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-image-registry-private-configuration\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.086256 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.086237 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-trusted-ca\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.086498 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.086457 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/940db99f-f328-4608-b40e-31243191d438-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-pkf7s\" (UID: \"940db99f-f328-4608-b40e-31243191d438\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" Jul 06 19:25:25.086498 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.086476 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4eee88bf-1be6-492f-82eb-b64464b0199e-cert\") pod \"ingress-canary-hfr9w\" (UID: \"4eee88bf-1be6-492f-82eb-b64464b0199e\") " pod="openshift-ingress-canary/ingress-canary-hfr9w" Jul 06 19:25:25.086848 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.086805 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/239c95d1-5bdb-4078-ad6b-9eb593957bb6-serving-cert\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.088821 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.088780 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-stats-auth\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:25.089116 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.089098 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-serving-cert\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:25.089198 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.089101 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-default-certificate\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:25.096942 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.096872 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-bound-sa-token\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.097157 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.097133 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6b2q\" (UniqueName: \"kubernetes.io/projected/5bd27dda-064e-48ba-8ae7-d5a14b609bd3-kube-api-access-t6b2q\") pod \"console-operator-68b757fb67-6cqbs\" (UID: \"5bd27dda-064e-48ba-8ae7-d5a14b609bd3\") " pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:25.097235 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.097143 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jrjd\" (UniqueName: \"kubernetes.io/projected/2369af8d-444e-488a-97c3-692ee5138a01-kube-api-access-6jrjd\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:25.097747 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.097725 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcdzg\" (UniqueName: \"kubernetes.io/projected/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-kube-api-access-dcdzg\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:25.097896 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.097874 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbc5g\" (UniqueName: \"kubernetes.io/projected/2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112-kube-api-access-pbc5g\") pod \"kube-storage-version-migrator-operator-846bf668f7-6qvnx\" (UID: \"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" Jul 06 19:25:25.099228 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.099209 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf9pr\" (UniqueName: \"kubernetes.io/projected/940db99f-f328-4608-b40e-31243191d438-kube-api-access-tf9pr\") pod \"service-ca-operator-5d46d9ffd4-pkf7s\" (UID: \"940db99f-f328-4608-b40e-31243191d438\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" Jul 06 19:25:25.099304 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.099288 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqf6l\" (UniqueName: \"kubernetes.io/projected/752bf161-7c8b-4af8-9d5d-3a7f9260e693-kube-api-access-mqf6l\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:25.100104 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.100084 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs44x\" (UniqueName: \"kubernetes.io/projected/7f1c75a3-c731-4050-82ba-444e5f4d5a9a-kube-api-access-vs44x\") pod \"volume-data-source-validator-5d456fdbcd-k6bbz\" (UID: \"7f1c75a3-c731-4050-82ba-444e5f4d5a9a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-k6bbz" Jul 06 19:25:25.101739 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.101717 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7cr7\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-kube-api-access-j7cr7\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.102002 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.101893 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvjtg\" (UniqueName: \"kubernetes.io/projected/55445c8d-30b3-47cd-9479-1ca223c726e5-kube-api-access-gvjtg\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:25.102372 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.102348 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plzrv\" (UniqueName: \"kubernetes.io/projected/239c95d1-5bdb-4078-ad6b-9eb593957bb6-kube-api-access-plzrv\") pod \"insights-operator-5c847dcfc4-rg9vj\" (UID: \"239c95d1-5bdb-4078-ad6b-9eb593957bb6\") " pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.102449 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.102397 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc9k4\" (UniqueName: \"kubernetes.io/projected/4eee88bf-1be6-492f-82eb-b64464b0199e-kube-api-access-jc9k4\") pod \"ingress-canary-hfr9w\" (UID: \"4eee88bf-1be6-492f-82eb-b64464b0199e\") " pod="openshift-ingress-canary/ingress-canary-hfr9w" Jul 06 19:25:25.143143 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.143113 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-k6bbz" Jul 06 19:25:25.156926 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.156883 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" Jul 06 19:25:25.166418 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.166395 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" Jul 06 19:25:25.169067 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.169014 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:25.171785 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.171756 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:25:25.172029 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.171761 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:25:25.172152 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.171773 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jkxjr\"" Jul 06 19:25:25.182450 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.182349 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:25.186504 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.186470 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:25.207815 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.207738 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" Jul 06 19:25:25.221802 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.221774 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hfr9w" Jul 06 19:25:25.391236 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.390746 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-k6bbz"] Jul 06 19:25:25.392244 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.392205 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-rg9vj"] Jul 06 19:25:25.398263 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:25.398151 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239c95d1_5bdb_4078_ad6b_9eb593957bb6.slice/crio-eb2e7ac61be51b51c12fb94fa502b41229a7e9e3a12b298bcbcf893086bb1237 WatchSource:0}: Error finding container eb2e7ac61be51b51c12fb94fa502b41229a7e9e3a12b298bcbcf893086bb1237: Status 404 returned error can't find the container with id eb2e7ac61be51b51c12fb94fa502b41229a7e9e3a12b298bcbcf893086bb1237 Jul 06 19:25:25.404188 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.404053 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" event={"ID":"239c95d1-5bdb-4078-ad6b-9eb593957bb6","Type":"ContainerStarted","Data":"eb2e7ac61be51b51c12fb94fa502b41229a7e9e3a12b298bcbcf893086bb1237"} Jul 06 19:25:25.405550 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.405522 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-k6bbz" event={"ID":"7f1c75a3-c731-4050-82ba-444e5f4d5a9a","Type":"ContainerStarted","Data":"b40427a32e3df5b5dc2625a966e44aa047845a94e8f8809984b37a3fe3c4d0e6"} Jul 06 19:25:25.419177 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.419143 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx"] Jul 06 19:25:25.421157 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:25.421127 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2512b8e7_36fe_4e92_9dc4_c1e0ae6d1112.slice/crio-71aa02fad94186a5da9b26996c77c98dcb93ebf787b213307aae03237d3b8418 WatchSource:0}: Error finding container 71aa02fad94186a5da9b26996c77c98dcb93ebf787b213307aae03237d3b8418: Status 404 returned error can't find the container with id 71aa02fad94186a5da9b26996c77c98dcb93ebf787b213307aae03237d3b8418 Jul 06 19:25:25.433478 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.433449 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-6cqbs"] Jul 06 19:25:25.436294 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:25.436262 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd27dda_064e_48ba_8ae7_d5a14b609bd3.slice/crio-e28310a30d23f69ae19e08be640e2279cdd41804b8c80ed44876bbaf94cdc7d4 WatchSource:0}: Error finding container e28310a30d23f69ae19e08be640e2279cdd41804b8c80ed44876bbaf94cdc7d4: Status 404 returned error can't find the container with id e28310a30d23f69ae19e08be640e2279cdd41804b8c80ed44876bbaf94cdc7d4 Jul 06 19:25:25.445631 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.445609 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-mtqzk"] Jul 06 19:25:25.449660 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.449640 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mtqzk" Jul 06 19:25:25.452196 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.452173 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qkk7z\"" Jul 06 19:25:25.459658 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.458622 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s"] Jul 06 19:25:25.462294 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:25.462268 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod940db99f_f328_4608_b40e_31243191d438.slice/crio-72d5223744c4e295b67c8264fcf0a2c9478a88e6c1345224ea9c5f3cc8b076f1 WatchSource:0}: Error finding container 72d5223744c4e295b67c8264fcf0a2c9478a88e6c1345224ea9c5f3cc8b076f1: Status 404 returned error can't find the container with id 72d5223744c4e295b67c8264fcf0a2c9478a88e6c1345224ea9c5f3cc8b076f1 Jul 06 19:25:25.462408 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.462384 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-mtqzk"] Jul 06 19:25:25.463534 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.463509 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hfr9w"] Jul 06 19:25:25.465355 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:25.465332 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eee88bf_1be6_492f_82eb_b64464b0199e.slice/crio-6b1b126a6419be27d2372e43dcba7553602f8a8f23e791c8e2c529b2cd6a5e34 WatchSource:0}: Error finding container 6b1b126a6419be27d2372e43dcba7553602f8a8f23e791c8e2c529b2cd6a5e34: Status 404 returned error can't find the container with id 6b1b126a6419be27d2372e43dcba7553602f8a8f23e791c8e2c529b2cd6a5e34 Jul 06 19:25:25.590372 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.590278 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:25.590546 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.590425 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:25.590546 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.590458 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:25.590546 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.590493 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:25:25.590702 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.590565 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:25:25.590702 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.590497 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:25.590702 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.590569 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls podName:752bf161-7c8b-4af8-9d5d-3a7f9260e693 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:26.590548981 +0000 UTC m=+37.095833997 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls") pod "dns-default-gd48l" (UID: "752bf161-7c8b-4af8-9d5d-3a7f9260e693") : secret "dns-default-metrics-tls" not found Jul 06 19:25:25.590702 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.590569 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:25.590702 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.590676 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:26.590657023 +0000 UTC m=+37.095942053 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:25.590702 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.590694 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls podName:1e15d9b3-599b-4f3b-8571-12b7a36a6d3e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:26.590683521 +0000 UTC m=+37.095968541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-nmz4s" (UID: "1e15d9b3-599b-4f3b-8571-12b7a36a6d3e") : secret "samples-operator-tls" not found Jul 06 19:25:25.590702 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.590709 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls podName:55445c8d-30b3-47cd-9479-1ca223c726e5 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:26.590700446 +0000 UTC m=+37.095985467 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lx4sd" (UID: "55445c8d-30b3-47cd-9479-1ca223c726e5") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:25.591059 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.590735 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxpjt\" (UniqueName: \"kubernetes.io/projected/d11e6f62-8617-4f80-af31-4c0eb3741146-kube-api-access-bxpjt\") pod \"network-check-source-989f8b8dc-mtqzk\" (UID: \"d11e6f62-8617-4f80-af31-4c0eb3741146\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mtqzk" Jul 06 19:25:25.591059 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.590774 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:25.591059 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.590804 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:25.591059 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.590869 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:25:25.591059 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.590878 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74b85d8d69-gq2gp: secret "image-registry-tls" not found Jul 06 19:25:25.591059 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.590900 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls podName:cd7bc946-cbb8-4966-b8c5-6edb16cc366e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:26.590892879 +0000 UTC m=+37.096177895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls") pod "image-registry-74b85d8d69-gq2gp" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e") : secret "image-registry-tls" not found Jul 06 19:25:25.591059 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.590951 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:25.591059 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:25.591023 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:26.591007433 +0000 UTC m=+37.096292449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : secret "router-metrics-certs-default" not found Jul 06 19:25:25.691941 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.691880 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpjt\" (UniqueName: \"kubernetes.io/projected/d11e6f62-8617-4f80-af31-4c0eb3741146-kube-api-access-bxpjt\") pod \"network-check-source-989f8b8dc-mtqzk\" (UID: \"d11e6f62-8617-4f80-af31-4c0eb3741146\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mtqzk" Jul 06 19:25:25.705540 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.705502 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxpjt\" (UniqueName: \"kubernetes.io/projected/d11e6f62-8617-4f80-af31-4c0eb3741146-kube-api-access-bxpjt\") pod \"network-check-source-989f8b8dc-mtqzk\" (UID: \"d11e6f62-8617-4f80-af31-4c0eb3741146\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mtqzk" Jul 06 19:25:25.763196 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.763160 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mtqzk" Jul 06 19:25:25.902714 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:25.902677 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-mtqzk"] Jul 06 19:25:25.906510 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:25.906477 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11e6f62_8617_4f80_af31_4c0eb3741146.slice/crio-da27d3708a4f39e48ff739e1ab94c1453b845c7d94681063ee5d78f8ebdfdc54 WatchSource:0}: Error finding container da27d3708a4f39e48ff739e1ab94c1453b845c7d94681063ee5d78f8ebdfdc54: Status 404 returned error can't find the container with id da27d3708a4f39e48ff739e1ab94c1453b845c7d94681063ee5d78f8ebdfdc54 Jul 06 19:25:26.170342 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.170089 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:26.170955 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.170933 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:26.173839 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.173228 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qhtzw\"" Jul 06 19:25:26.173839 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.173446 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:25:26.173839 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.173690 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:25:26.411638 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.411535 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" event={"ID":"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112","Type":"ContainerStarted","Data":"71aa02fad94186a5da9b26996c77c98dcb93ebf787b213307aae03237d3b8418"} Jul 06 19:25:26.417893 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.417847 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" event={"ID":"940db99f-f328-4608-b40e-31243191d438","Type":"ContainerStarted","Data":"72d5223744c4e295b67c8264fcf0a2c9478a88e6c1345224ea9c5f3cc8b076f1"} Jul 06 19:25:26.423118 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.422997 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" event={"ID":"5bd27dda-064e-48ba-8ae7-d5a14b609bd3","Type":"ContainerStarted","Data":"e28310a30d23f69ae19e08be640e2279cdd41804b8c80ed44876bbaf94cdc7d4"} Jul 06 19:25:26.425115 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.425039 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mtqzk" event={"ID":"d11e6f62-8617-4f80-af31-4c0eb3741146","Type":"ContainerStarted","Data":"da27d3708a4f39e48ff739e1ab94c1453b845c7d94681063ee5d78f8ebdfdc54"} Jul 06 19:25:26.426819 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.426751 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hfr9w" event={"ID":"4eee88bf-1be6-492f-82eb-b64464b0199e","Type":"ContainerStarted","Data":"6b1b126a6419be27d2372e43dcba7553602f8a8f23e791c8e2c529b2cd6a5e34"} Jul 06 19:25:26.550040 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.549705 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.602020 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.602068 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.602104 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.602136 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.602166 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.602209 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602381 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602448 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls podName:752bf161-7c8b-4af8-9d5d-3a7f9260e693 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:28.602427737 +0000 UTC m=+39.107712757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls") pod "dns-default-gd48l" (UID: "752bf161-7c8b-4af8-9d5d-3a7f9260e693") : secret "dns-default-metrics-tls" not found Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602515 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602550 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls podName:1e15d9b3-599b-4f3b-8571-12b7a36a6d3e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:28.602538878 +0000 UTC m=+39.107823893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-nmz4s" (UID: "1e15d9b3-599b-4f3b-8571-12b7a36a6d3e") : secret "samples-operator-tls" not found Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602617 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:28.60260752 +0000 UTC m=+39.107892537 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:26.602988 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602698 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:26.603730 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602740 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:28.602727482 +0000 UTC m=+39.108012510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : secret "router-metrics-certs-default" not found Jul 06 19:25:26.603730 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602797 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:26.603730 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602827 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls podName:55445c8d-30b3-47cd-9479-1ca223c726e5 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:28.602817115 +0000 UTC m=+39.108102144 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lx4sd" (UID: "55445c8d-30b3-47cd-9479-1ca223c726e5") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:26.603730 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602879 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:25:26.603730 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602889 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74b85d8d69-gq2gp: secret "image-registry-tls" not found Jul 06 19:25:26.603730 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:26.602938 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls podName:cd7bc946-cbb8-4966-b8c5-6edb16cc366e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:28.602928341 +0000 UTC m=+39.108213376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls") pod "image-registry-74b85d8d69-gq2gp" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e") : secret "image-registry-tls" not found Jul 06 19:25:26.660006 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:26.659970 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:27.436301 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:27.435942 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-k6bbz" event={"ID":"7f1c75a3-c731-4050-82ba-444e5f4d5a9a","Type":"ContainerStarted","Data":"8c19f45447711ca416e6981fb03561a0d1f30975e0aa2bce385f462ed9b93f50"} Jul 06 19:25:27.454568 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:27.453962 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-k6bbz" podStartSLOduration=11.731739859 podStartE2EDuration="13.453936882s" totalImagesPullingTime="1.722197023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:27.452413445 +0000 UTC m=+37.957698482" watchObservedRunningTime="2026-07-06 19:25:27.453936882 +0000 UTC m=+37.959221925" Jul 06 19:25:27.600513 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:27.600457 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:27.666249 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:27.666197 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:28.622442 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:28.622403 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:28.622442 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:28.622451 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:28.622488 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:28.622521 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.622547 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.622590 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.622611 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:32.622589343 +0000 UTC m=+43.127874366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.622618 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.622644 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.622666 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74b85d8d69-gq2gp: secret "image-registry-tls" not found Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:28.622552 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.622644 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls podName:55445c8d-30b3-47cd-9479-1ca223c726e5 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:32.62262731 +0000 UTC m=+43.127912343 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lx4sd" (UID: "55445c8d-30b3-47cd-9479-1ca223c726e5") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.622749 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls podName:1e15d9b3-599b-4f3b-8571-12b7a36a6d3e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:32.622732936 +0000 UTC m=+43.128017956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-nmz4s" (UID: "1e15d9b3-599b-4f3b-8571-12b7a36a6d3e") : secret "samples-operator-tls" not found Jul 06 19:25:28.623048 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.622773 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:32.622763389 +0000 UTC m=+43.128048407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : secret "router-metrics-certs-default" not found Jul 06 19:25:28.623557 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.622791 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls podName:cd7bc946-cbb8-4966-b8c5-6edb16cc366e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:32.622782702 +0000 UTC m=+43.128067720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls") pod "image-registry-74b85d8d69-gq2gp" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e") : secret "image-registry-tls" not found Jul 06 19:25:28.623557 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:28.622839 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:28.623557 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.623012 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:25:28.623557 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:28.623057 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls podName:752bf161-7c8b-4af8-9d5d-3a7f9260e693 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:32.623045102 +0000 UTC m=+43.128330124 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls") pod "dns-default-gd48l" (UID: "752bf161-7c8b-4af8-9d5d-3a7f9260e693") : secret "dns-default-metrics-tls" not found Jul 06 19:25:30.285930 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.285875 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:30.355700 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.355650 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:30.438835 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.438801 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4"] Jul 06 19:25:30.469096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.469065 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4"] Jul 06 19:25:30.469240 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.469193 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:25:30.471615 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.471573 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 06 19:25:30.471615 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.471604 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2fj6l\"" Jul 06 19:25:30.471821 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.471683 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 06 19:25:30.542022 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.541935 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:25:30.542170 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.542103 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94235910-e2b4-4f48-9c04-9a01df47b39e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:25:30.643496 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.643456 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:25:30.643674 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.643564 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94235910-e2b4-4f48-9c04-9a01df47b39e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:25:30.643674 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:30.643628 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 19:25:30.643762 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:30.643698 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert podName:94235910-e2b4-4f48-9c04-9a01df47b39e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:31.143680173 +0000 UTC m=+41.648965212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fqdb4" (UID: "94235910-e2b4-4f48-9c04-9a01df47b39e") : secret "networking-console-plugin-cert" not found Jul 06 19:25:30.644488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:30.644468 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94235910-e2b4-4f48-9c04-9a01df47b39e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:25:31.148393 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:31.148351 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:25:31.148595 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:31.148524 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 19:25:31.148682 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:31.148609 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert podName:94235910-e2b4-4f48-9c04-9a01df47b39e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:32.148590704 +0000 UTC m=+42.653875720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fqdb4" (UID: "94235910-e2b4-4f48-9c04-9a01df47b39e") : secret "networking-console-plugin-cert" not found Jul 06 19:25:32.157286 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:32.157235 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:25:32.157739 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.157412 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 19:25:32.157739 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.157483 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert podName:94235910-e2b4-4f48-9c04-9a01df47b39e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:34.157466429 +0000 UTC m=+44.662751448 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fqdb4" (UID: "94235910-e2b4-4f48-9c04-9a01df47b39e") : secret "networking-console-plugin-cert" not found Jul 06 19:25:32.665171 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:32.665131 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:32.665171 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:32.665174 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:32.665426 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:32.665211 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:32.665426 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:32.665244 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:32.665426 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:32.665277 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:32.665426 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:32.665335 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:32.665426 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665347 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:32.665426 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665374 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:32.665426 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665401 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:25:32.665426 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665420 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:25:32.665755 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665357 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:40.665336586 +0000 UTC m=+51.170621616 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:32.665755 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665463 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:25:32.665755 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665473 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:40.665457049 +0000 UTC m=+51.170742067 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : secret "router-metrics-certs-default" not found Jul 06 19:25:32.665755 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665475 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74b85d8d69-gq2gp: secret "image-registry-tls" not found Jul 06 19:25:32.665755 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665493 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls podName:55445c8d-30b3-47cd-9479-1ca223c726e5 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:40.665482466 +0000 UTC m=+51.170767488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lx4sd" (UID: "55445c8d-30b3-47cd-9479-1ca223c726e5") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:32.665755 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665511 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls podName:752bf161-7c8b-4af8-9d5d-3a7f9260e693 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:40.665502466 +0000 UTC m=+51.170787482 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls") pod "dns-default-gd48l" (UID: "752bf161-7c8b-4af8-9d5d-3a7f9260e693") : secret "dns-default-metrics-tls" not found Jul 06 19:25:32.665755 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665533 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls podName:1e15d9b3-599b-4f3b-8571-12b7a36a6d3e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:40.665527453 +0000 UTC m=+51.170812469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-nmz4s" (UID: "1e15d9b3-599b-4f3b-8571-12b7a36a6d3e") : secret "samples-operator-tls" not found Jul 06 19:25:32.665755 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:32.665543 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls podName:cd7bc946-cbb8-4966-b8c5-6edb16cc366e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:40.665538397 +0000 UTC m=+51.170823412 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls") pod "image-registry-74b85d8d69-gq2gp" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e") : secret "image-registry-tls" not found Jul 06 19:25:33.616077 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:33.616039 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:33.713603 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:33.713574 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:33.779592 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:33.779505 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:34.179021 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:34.178949 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:25:34.179168 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:34.179061 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 19:25:34.179168 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:34.179128 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert podName:94235910-e2b4-4f48-9c04-9a01df47b39e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:38.179108859 +0000 UTC m=+48.684393878 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fqdb4" (UID: "94235910-e2b4-4f48-9c04-9a01df47b39e") : secret "networking-console-plugin-cert" not found Jul 06 19:25:34.214347 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:34.214308 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:34.291666 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:34.291620 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:34.391575 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:34.391541 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:34.591782 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:34.591743 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:34.965872 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:34.965846 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:35.065467 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.065433 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:35.460201 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.459613 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" event={"ID":"239c95d1-5bdb-4078-ad6b-9eb593957bb6","Type":"ContainerStarted","Data":"ef2403078f58ff33f1c114f029aacc1eec88b8d335573e0424bc8fbe4d9f0d52"} Jul 06 19:25:35.461720 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.461666 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" event={"ID":"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112","Type":"ContainerStarted","Data":"0565ece6125e14cd409104123ce8993222aadde9ccd8fbebec71738866c724c2"} Jul 06 19:25:35.463464 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.463418 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" event={"ID":"940db99f-f328-4608-b40e-31243191d438","Type":"ContainerStarted","Data":"962b0773b02d5f52f84e0173229d91c73864a2ce8ad1801c7bda40947b5cd65e"} Jul 06 19:25:35.465203 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.465175 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/0.log" Jul 06 19:25:35.465318 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.465211 2572 generic.go:356] "Generic (PLEG): container finished" podID="5bd27dda-064e-48ba-8ae7-d5a14b609bd3" containerID="ef772aef2daa4636f4bf68a68d07aa0794410d4bbc797161ef52929b9b94b96d" exitCode=255 Jul 06 19:25:35.465458 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.465435 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" event={"ID":"5bd27dda-064e-48ba-8ae7-d5a14b609bd3","Type":"ContainerDied","Data":"ef772aef2daa4636f4bf68a68d07aa0794410d4bbc797161ef52929b9b94b96d"} Jul 06 19:25:35.465547 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.465516 2572 scope.go:117] "RemoveContainer" containerID="ef772aef2daa4636f4bf68a68d07aa0794410d4bbc797161ef52929b9b94b96d" Jul 06 19:25:35.468603 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.468520 2572 generic.go:356] "Generic (PLEG): container finished" podID="e817f8b8-1d4a-445d-91c4-6c6d77d2bd28" containerID="731584025e4687203f1b682eeaa2d37ae150a044ae0d371cc6eb832ae0531340" exitCode=0 Jul 06 19:25:35.468703 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.468603 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqg6b" event={"ID":"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28","Type":"ContainerDied","Data":"731584025e4687203f1b682eeaa2d37ae150a044ae0d371cc6eb832ae0531340"} Jul 06 19:25:35.470853 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.470830 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mtqzk" event={"ID":"d11e6f62-8617-4f80-af31-4c0eb3741146","Type":"ContainerStarted","Data":"6c85f78fdf3c373b906722039e88fa2416ec7500961b1bded619ca7d0a89d102"} Jul 06 19:25:35.472952 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.472897 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hfr9w" event={"ID":"4eee88bf-1be6-492f-82eb-b64464b0199e","Type":"ContainerStarted","Data":"c0dd4965bfe038b004e3e7289692be521578314d95e0f523febfe8632a4daa60"} Jul 06 19:25:35.491822 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.490734 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" podStartSLOduration=12.136118996 podStartE2EDuration="21.490713172s" totalImagesPullingTime="9.354594176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:35.488663377 +0000 UTC m=+45.993948416" watchObservedRunningTime="2026-07-06 19:25:35.490713172 +0000 UTC m=+45.995998227" Jul 06 19:25:35.542992 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.542931 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mtqzk" podStartSLOduration=1.334318986 podStartE2EDuration="10.542891471s" totalImagesPullingTime="9.208572485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:35.542175608 +0000 UTC m=+46.047460646" watchObservedRunningTime="2026-07-06 19:25:35.542891471 +0000 UTC m=+46.048176509" Jul 06 19:25:35.598068 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.596630 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hfr9w" podStartSLOduration=2.26726051 podStartE2EDuration="11.59660879s" totalImagesPullingTime="9.32934828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:35.574565493 +0000 UTC m=+46.079850531" watchObservedRunningTime="2026-07-06 19:25:35.59660879 +0000 UTC m=+46.101893820" Jul 06 19:25:35.598068 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.597042 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" podStartSLOduration=12.271416187 podStartE2EDuration="21.597031969s" totalImagesPullingTime="9.325615782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:35.596703235 +0000 UTC m=+46.101988274" watchObservedRunningTime="2026-07-06 19:25:35.597031969 +0000 UTC m=+46.102317008" Jul 06 19:25:35.622079 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.622016 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" podStartSLOduration=12.255854201 podStartE2EDuration="21.621993131s" totalImagesPullingTime="9.36613893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:35.620328275 +0000 UTC m=+46.125613314" watchObservedRunningTime="2026-07-06 19:25:35.621993131 +0000 UTC m=+46.127278170" Jul 06 19:25:35.776727 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:35.776636 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:36.184089 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.184002 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:36.257486 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.257449 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:36.470048 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.470012 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x6xzx"] Jul 06 19:25:36.479614 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.479538 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:25:36.481419 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.480168 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/0.log" Jul 06 19:25:36.481419 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.480205 2572 generic.go:356] "Generic (PLEG): container finished" podID="5bd27dda-064e-48ba-8ae7-d5a14b609bd3" containerID="ee60c6941ffd979dceab6f834e524fe1d41da0cc5f8748d39d5e0e330f2199a6" exitCode=255 Jul 06 19:25:36.481419 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.480621 2572 scope.go:117] "RemoveContainer" containerID="ee60c6941ffd979dceab6f834e524fe1d41da0cc5f8748d39d5e0e330f2199a6" Jul 06 19:25:36.481419 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:36.480804 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-6cqbs_openshift-console-operator(5bd27dda-064e-48ba-8ae7-d5a14b609bd3)\"" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" podUID="5bd27dda-064e-48ba-8ae7-d5a14b609bd3" Jul 06 19:25:36.486548 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.486493 2572 generic.go:356] "Generic (PLEG): container finished" podID="e817f8b8-1d4a-445d-91c4-6c6d77d2bd28" containerID="0ea2604b8cc368b954d97ba64ce75c115f03e5fdaa05a4a3bd098c808610655d" exitCode=0 Jul 06 19:25:36.492312 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.489593 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" event={"ID":"5bd27dda-064e-48ba-8ae7-d5a14b609bd3","Type":"ContainerDied","Data":"ee60c6941ffd979dceab6f834e524fe1d41da0cc5f8748d39d5e0e330f2199a6"} Jul 06 19:25:36.492312 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.489629 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqg6b" event={"ID":"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28","Type":"ContainerDied","Data":"0ea2604b8cc368b954d97ba64ce75c115f03e5fdaa05a4a3bd098c808610655d"} Jul 06 19:25:36.492312 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.489663 2572 scope.go:117] "RemoveContainer" containerID="ef772aef2daa4636f4bf68a68d07aa0794410d4bbc797161ef52929b9b94b96d" Jul 06 19:25:36.492312 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.489847 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.492312 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.489853 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x6xzx"] Jul 06 19:25:36.492566 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.492346 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 19:25:36.492612 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.492574 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 19:25:36.492826 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.492808 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f282m\"" Jul 06 19:25:36.518695 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.518652 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:36.580075 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.580022 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:36.602612 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.602578 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.602777 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.602751 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.602934 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.602901 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87921692-d802-479d-a041-a5ba5b5f78af-crio-socket\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.603013 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.603000 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87921692-d802-479d-a041-a5ba5b5f78af-data-volume\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.603287 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.603268 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfzts\" (UniqueName: \"kubernetes.io/projected/87921692-d802-479d-a041-a5ba5b5f78af-kube-api-access-xfzts\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.704683 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.704650 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfzts\" (UniqueName: \"kubernetes.io/projected/87921692-d802-479d-a041-a5ba5b5f78af-kube-api-access-xfzts\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.704878 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.704714 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.704878 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.704751 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.704878 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.704811 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87921692-d802-479d-a041-a5ba5b5f78af-crio-socket\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.704878 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.704845 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87921692-d802-479d-a041-a5ba5b5f78af-data-volume\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.705112 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:36.704920 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:36.705112 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:36.704996 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls podName:87921692-d802-479d-a041-a5ba5b5f78af nodeName:}" failed. No retries permitted until 2026-07-06 19:25:37.204976017 +0000 UTC m=+47.710261047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x6xzx" (UID: "87921692-d802-479d-a041-a5ba5b5f78af") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:36.705112 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.705048 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/87921692-d802-479d-a041-a5ba5b5f78af-crio-socket\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.705112 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:36.704919 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:36.705263 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:36.705133 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm podName:87921692-d802-479d-a041-a5ba5b5f78af nodeName:}" failed. No retries permitted until 2026-07-06 19:25:37.205114584 +0000 UTC m=+47.710399617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x6xzx" (UID: "87921692-d802-479d-a041-a5ba5b5f78af") : configmap "kube-rbac-proxy" not found Jul 06 19:25:36.705263 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.705191 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/87921692-d802-479d-a041-a5ba5b5f78af-data-volume\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.715740 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.715712 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfzts\" (UniqueName: \"kubernetes.io/projected/87921692-d802-479d-a041-a5ba5b5f78af-kube-api-access-xfzts\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:36.895998 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.895890 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:36.941302 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.941263 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j"] Jul 06 19:25:36.956326 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.956291 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j" Jul 06 19:25:36.957240 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.957199 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j"] Jul 06 19:25:36.959472 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.959438 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 19:25:36.959607 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.959438 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 19:25:36.959832 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.959816 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ch95p\"" Jul 06 19:25:36.994765 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:36.994731 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:37.008131 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.008092 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svknk\" (UniqueName: \"kubernetes.io/projected/4a7e8265-6ce1-42c3-b589-43b0185cee0d-kube-api-access-svknk\") pod \"migrator-d57d58cc9-5cr4j\" (UID: \"4a7e8265-6ce1-42c3-b589-43b0185cee0d\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j" Jul 06 19:25:37.082520 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.082481 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:37.108819 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.108782 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svknk\" (UniqueName: \"kubernetes.io/projected/4a7e8265-6ce1-42c3-b589-43b0185cee0d-kube-api-access-svknk\") pod \"migrator-d57d58cc9-5cr4j\" (UID: \"4a7e8265-6ce1-42c3-b589-43b0185cee0d\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j" Jul 06 19:25:37.122610 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.122580 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svknk\" (UniqueName: \"kubernetes.io/projected/4a7e8265-6ce1-42c3-b589-43b0185cee0d-kube-api-access-svknk\") pod \"migrator-d57d58cc9-5cr4j\" (UID: \"4a7e8265-6ce1-42c3-b589-43b0185cee0d\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j" Jul 06 19:25:37.210205 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.210170 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:37.210618 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.210233 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:37.210618 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:37.210303 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:37.210618 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:37.210329 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:37.210618 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:37.210371 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm podName:87921692-d802-479d-a041-a5ba5b5f78af nodeName:}" failed. No retries permitted until 2026-07-06 19:25:38.210355566 +0000 UTC m=+48.715640582 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x6xzx" (UID: "87921692-d802-479d-a041-a5ba5b5f78af") : configmap "kube-rbac-proxy" not found Jul 06 19:25:37.210618 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:37.210392 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls podName:87921692-d802-479d-a041-a5ba5b5f78af nodeName:}" failed. No retries permitted until 2026-07-06 19:25:38.210378041 +0000 UTC m=+48.715663056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x6xzx" (UID: "87921692-d802-479d-a041-a5ba5b5f78af") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:37.268195 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.268159 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j" Jul 06 19:25:37.432029 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.431995 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j"] Jul 06 19:25:37.435108 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:37.435072 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a7e8265_6ce1_42c3_b589_43b0185cee0d.slice/crio-097c97f7ab6bbdd4601d398b599a89bf7825450c8e176223216937ae7fd1ab7d WatchSource:0}: Error finding container 097c97f7ab6bbdd4601d398b599a89bf7825450c8e176223216937ae7fd1ab7d: Status 404 returned error can't find the container with id 097c97f7ab6bbdd4601d398b599a89bf7825450c8e176223216937ae7fd1ab7d Jul 06 19:25:37.466708 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.466617 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:37.491550 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.491525 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:25:37.491948 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.491900 2572 scope.go:117] "RemoveContainer" containerID="ee60c6941ffd979dceab6f834e524fe1d41da0cc5f8748d39d5e0e330f2199a6" Jul 06 19:25:37.492117 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:37.492099 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-6cqbs_openshift-console-operator(5bd27dda-064e-48ba-8ae7-d5a14b609bd3)\"" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" podUID="5bd27dda-064e-48ba-8ae7-d5a14b609bd3" Jul 06 19:25:37.495046 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.495024 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqg6b" event={"ID":"e817f8b8-1d4a-445d-91c4-6c6d77d2bd28","Type":"ContainerStarted","Data":"e1ad90a079b12261cab4d6618f6b0c87340899a2c3ecc10fed4a72ea7a4e8979"} Jul 06 19:25:37.496126 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.496108 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j" event={"ID":"4a7e8265-6ce1-42c3-b589-43b0185cee0d","Type":"ContainerStarted","Data":"097c97f7ab6bbdd4601d398b599a89bf7825450c8e176223216937ae7fd1ab7d"} Jul 06 19:25:37.538326 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.538269 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zqg6b" podStartSLOduration=7.509716276 podStartE2EDuration="47.538252713s" totalImagesPullingTime="40.028536437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:37.537844037 +0000 UTC m=+48.043129075" watchObservedRunningTime="2026-07-06 19:25:37.538252713 +0000 UTC m=+48.043537750" Jul 06 19:25:37.542512 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.542484 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:37.679884 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.679854 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ljx8t_68f9a6e3-08f5-4aa1-aba6-f07ff314e006/dns-node-resolver/0.log" Jul 06 19:25:37.680226 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.680200 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:37.779970 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:37.779854 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:38.219330 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:38.219304 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:38.219714 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:38.219345 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:25:38.219714 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:38.219367 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:38.219714 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:38.219431 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:38.219714 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:38.219480 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 19:25:38.219714 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:38.219530 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:38.219714 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:38.219487 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm podName:87921692-d802-479d-a041-a5ba5b5f78af nodeName:}" failed. No retries permitted until 2026-07-06 19:25:40.219473439 +0000 UTC m=+50.724758455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x6xzx" (UID: "87921692-d802-479d-a041-a5ba5b5f78af") : configmap "kube-rbac-proxy" not found Jul 06 19:25:38.219714 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:38.219574 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert podName:94235910-e2b4-4f48-9c04-9a01df47b39e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:46.219554828 +0000 UTC m=+56.724839851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fqdb4" (UID: "94235910-e2b4-4f48-9c04-9a01df47b39e") : secret "networking-console-plugin-cert" not found Jul 06 19:25:38.219714 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:38.219694 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls podName:87921692-d802-479d-a041-a5ba5b5f78af nodeName:}" failed. No retries permitted until 2026-07-06 19:25:40.219673027 +0000 UTC m=+50.724958047 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x6xzx" (UID: "87921692-d802-479d-a041-a5ba5b5f78af") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:38.874127 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:38.874099 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z2r5s_09767752-33c4-4574-b852-5d40a6c8929f/node-ca/0.log" Jul 06 19:25:39.244016 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.243971 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:39.272836 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.272806 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hfr9w_4eee88bf-1be6-492f-82eb-b64464b0199e/serve-healthcheck-canary/0.log" Jul 06 19:25:39.359891 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.359858 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-6zk6k"] Jul 06 19:25:39.381986 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.381951 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-6zk6k"] Jul 06 19:25:39.382140 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.382010 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-6zk6k" Jul 06 19:25:39.384635 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.384604 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 19:25:39.384783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.384612 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 19:25:39.384783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.384690 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 19:25:39.385660 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.385642 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6mkw2\"" Jul 06 19:25:39.385782 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.385695 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 19:25:39.407362 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.407324 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:39.432307 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.432275 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6c9164a-4de1-46c7-91b7-dc3995310db0-signing-key\") pod \"service-ca-758f599c5-6zk6k\" (UID: \"e6c9164a-4de1-46c7-91b7-dc3995310db0\") " pod="openshift-service-ca/service-ca-758f599c5-6zk6k" Jul 06 19:25:39.432485 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.432337 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6c9164a-4de1-46c7-91b7-dc3995310db0-signing-cabundle\") pod \"service-ca-758f599c5-6zk6k\" (UID: \"e6c9164a-4de1-46c7-91b7-dc3995310db0\") " pod="openshift-service-ca/service-ca-758f599c5-6zk6k" Jul 06 19:25:39.432604 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.432570 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9zkp\" (UniqueName: \"kubernetes.io/projected/e6c9164a-4de1-46c7-91b7-dc3995310db0-kube-api-access-q9zkp\") pod \"service-ca-758f599c5-6zk6k\" (UID: \"e6c9164a-4de1-46c7-91b7-dc3995310db0\") " pod="openshift-service-ca/service-ca-758f599c5-6zk6k" Jul 06 19:25:39.480056 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.480023 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:39.533700 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.533678 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6c9164a-4de1-46c7-91b7-dc3995310db0-signing-key\") pod \"service-ca-758f599c5-6zk6k\" (UID: \"e6c9164a-4de1-46c7-91b7-dc3995310db0\") " pod="openshift-service-ca/service-ca-758f599c5-6zk6k" Jul 06 19:25:39.533781 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.533725 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6c9164a-4de1-46c7-91b7-dc3995310db0-signing-cabundle\") pod \"service-ca-758f599c5-6zk6k\" (UID: \"e6c9164a-4de1-46c7-91b7-dc3995310db0\") " pod="openshift-service-ca/service-ca-758f599c5-6zk6k" Jul 06 19:25:39.534000 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.533977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9zkp\" (UniqueName: \"kubernetes.io/projected/e6c9164a-4de1-46c7-91b7-dc3995310db0-kube-api-access-q9zkp\") pod \"service-ca-758f599c5-6zk6k\" (UID: \"e6c9164a-4de1-46c7-91b7-dc3995310db0\") " pod="openshift-service-ca/service-ca-758f599c5-6zk6k" Jul 06 19:25:39.534408 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.534388 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6c9164a-4de1-46c7-91b7-dc3995310db0-signing-cabundle\") pod \"service-ca-758f599c5-6zk6k\" (UID: \"e6c9164a-4de1-46c7-91b7-dc3995310db0\") " pod="openshift-service-ca/service-ca-758f599c5-6zk6k" Jul 06 19:25:39.536399 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.536369 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6c9164a-4de1-46c7-91b7-dc3995310db0-signing-key\") pod \"service-ca-758f599c5-6zk6k\" (UID: \"e6c9164a-4de1-46c7-91b7-dc3995310db0\") " pod="openshift-service-ca/service-ca-758f599c5-6zk6k" Jul 06 19:25:39.543541 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.543510 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9zkp\" (UniqueName: \"kubernetes.io/projected/e6c9164a-4de1-46c7-91b7-dc3995310db0-kube-api-access-q9zkp\") pod \"service-ca-758f599c5-6zk6k\" (UID: \"e6c9164a-4de1-46c7-91b7-dc3995310db0\") " pod="openshift-service-ca/service-ca-758f599c5-6zk6k" Jul 06 19:25:39.691870 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.691830 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-6zk6k" Jul 06 19:25:39.851365 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:39.851342 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-6zk6k"] Jul 06 19:25:39.853316 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:39.853288 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6c9164a_4de1_46c7_91b7_dc3995310db0.slice/crio-103374cd513dd9b99b2687933569bb2a40c692e74c0610c864d2157f0e8f7920 WatchSource:0}: Error finding container 103374cd513dd9b99b2687933569bb2a40c692e74c0610c864d2157f0e8f7920: Status 404 returned error can't find the container with id 103374cd513dd9b99b2687933569bb2a40c692e74c0610c864d2157f0e8f7920 Jul 06 19:25:40.210938 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.210895 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:40.240305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.240276 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:40.240439 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.240416 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:40.240479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.240434 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:40.240512 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.240497 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm podName:87921692-d802-479d-a041-a5ba5b5f78af nodeName:}" failed. No retries permitted until 2026-07-06 19:25:44.240476549 +0000 UTC m=+54.745761570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x6xzx" (UID: "87921692-d802-479d-a041-a5ba5b5f78af") : configmap "kube-rbac-proxy" not found Jul 06 19:25:40.240557 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.240535 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:40.240591 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.240582 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls podName:87921692-d802-479d-a041-a5ba5b5f78af nodeName:}" failed. No retries permitted until 2026-07-06 19:25:44.240566989 +0000 UTC m=+54.745852009 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x6xzx" (UID: "87921692-d802-479d-a041-a5ba5b5f78af") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:40.512592 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.512483 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j" event={"ID":"4a7e8265-6ce1-42c3-b589-43b0185cee0d","Type":"ContainerStarted","Data":"0aeb76eb7950d7d64232a9b83c62e962bf1974084be604f83e4f7023d42d93a3"} Jul 06 19:25:40.512592 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.512582 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j" event={"ID":"4a7e8265-6ce1-42c3-b589-43b0185cee0d","Type":"ContainerStarted","Data":"941d987bd6d73164270a2fd1a52151c7896a26d86fa4a3b033b4a36254ae5eaf"} Jul 06 19:25:40.514060 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.514030 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-6zk6k" event={"ID":"e6c9164a-4de1-46c7-91b7-dc3995310db0","Type":"ContainerStarted","Data":"80e19d3ecb510407115798b23903dc95a355854889a9c5124c4c54357178cf30"} Jul 06 19:25:40.514183 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.514079 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-6zk6k" event={"ID":"e6c9164a-4de1-46c7-91b7-dc3995310db0","Type":"ContainerStarted","Data":"103374cd513dd9b99b2687933569bb2a40c692e74c0610c864d2157f0e8f7920"} Jul 06 19:25:40.533268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.533204 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5cr4j" podStartSLOduration=2.461030776 podStartE2EDuration="4.533184933s" totalImagesPullingTime="2.072154157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:40.53178397 +0000 UTC m=+51.037069010" watchObservedRunningTime="2026-07-06 19:25:40.533184933 +0000 UTC m=+51.038470039" Jul 06 19:25:40.549953 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.549878 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-6zk6k" podStartSLOduration=1.549859893 podStartE2EDuration="1.549859893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:40.548776515 +0000 UTC m=+51.054061554" watchObservedRunningTime="2026-07-06 19:25:40.549859893 +0000 UTC m=+51.055144930" Jul 06 19:25:40.745745 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.745706 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:40.745745 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.745747 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.745771 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.745801 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.745819 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.745887 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.745924 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.746003 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls podName:1e15d9b3-599b-4f3b-8571-12b7a36a6d3e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:56.745982514 +0000 UTC m=+67.251267529 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-nmz4s" (UID: "1e15d9b3-599b-4f3b-8571-12b7a36a6d3e") : secret "samples-operator-tls" not found Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:40.745988 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.746024 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:56.746015114 +0000 UTC m=+67.251300130 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : configmap references non-existent config key: service-ca.crt Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.745937 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.746057 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls podName:55445c8d-30b3-47cd-9479-1ca223c726e5 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:56.746038414 +0000 UTC m=+67.251323449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-lx4sd" (UID: "55445c8d-30b3-47cd-9479-1ca223c726e5") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.746056 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.746064 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74b85d8d69-gq2gp: secret "image-registry-tls" not found Jul 06 19:25:40.746104 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.746101 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls podName:752bf161-7c8b-4af8-9d5d-3a7f9260e693 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:56.746090347 +0000 UTC m=+67.251375363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls") pod "dns-default-gd48l" (UID: "752bf161-7c8b-4af8-9d5d-3a7f9260e693") : secret "dns-default-metrics-tls" not found Jul 06 19:25:40.746534 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.745980 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:25:40.746534 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.746126 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls podName:cd7bc946-cbb8-4966-b8c5-6edb16cc366e nodeName:}" failed. No retries permitted until 2026-07-06 19:25:56.746115351 +0000 UTC m=+67.251400369 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls") pod "image-registry-74b85d8d69-gq2gp" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e") : secret "image-registry-tls" not found Jul 06 19:25:40.746534 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:40.746248 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs podName:2369af8d-444e-488a-97c3-692ee5138a01 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:56.746235135 +0000 UTC m=+67.251520151 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs") pod "router-default-756fc79d99-fvh2j" (UID: "2369af8d-444e-488a-97c3-692ee5138a01") : secret "router-metrics-certs-default" not found Jul 06 19:25:41.015092 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:41.015022 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:41.115885 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:41.115838 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:41.197525 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:41.197462 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:41.303503 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:41.303406 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:41.396958 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:41.396901 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:41.503651 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:41.503570 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:41.599224 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:41.599138 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:41.899355 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:41.899254 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:42.495970 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:42.495931 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:43.096402 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:43.096341 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:43.294934 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:43.294886 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:43.358517 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:43.358428 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:43.502163 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:43.502117 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:43.891832 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:43.891795 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:44.283276 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:44.283231 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:44.283689 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:44.283292 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:44.283689 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:44.283412 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:44.283689 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:44.283489 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls podName:87921692-d802-479d-a041-a5ba5b5f78af nodeName:}" failed. No retries permitted until 2026-07-06 19:25:52.283473938 +0000 UTC m=+62.788758954 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x6xzx" (UID: "87921692-d802-479d-a041-a5ba5b5f78af") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:44.283832 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:44.283813 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/87921692-d802-479d-a041-a5ba5b5f78af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:44.504178 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:44.504134 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:44.692039 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:44.691941 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:44.970990 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:44.970955 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:45.105660 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:45.105619 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:45.186928 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:45.186870 2572 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:45.187130 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:45.186950 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:25:45.187326 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:45.187312 2572 scope.go:117] "RemoveContainer" containerID="ee60c6941ffd979dceab6f834e524fe1d41da0cc5f8748d39d5e0e330f2199a6" Jul 06 19:25:45.187486 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:45.187468 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-6cqbs_openshift-console-operator(5bd27dda-064e-48ba-8ae7-d5a14b609bd3)\"" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" podUID="5bd27dda-064e-48ba-8ae7-d5a14b609bd3" Jul 06 19:25:45.300295 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:45.300201 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:45.707736 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:45.707649 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:45.772803 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:45.772763 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:45.898212 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:45.898168 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:46.302036 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:46.301987 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:25:46.302533 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:46.302132 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 19:25:46.302533 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:25:46.302226 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert podName:94235910-e2b4-4f48-9c04-9a01df47b39e nodeName:}" failed. No retries permitted until 2026-07-06 19:26:02.302204728 +0000 UTC m=+72.807489759 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fqdb4" (UID: "94235910-e2b4-4f48-9c04-9a01df47b39e") : secret "networking-console-plugin-cert" not found Jul 06 19:25:46.302961 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:46.302928 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:46.475268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:46.475237 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:46.573610 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:46.573503 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:46.896061 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:46.895960 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:47.395704 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:47.395668 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:47.515372 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:47.515342 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:47.696534 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:47.696445 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:47.887968 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:47.887937 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:48.109727 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:48.109690 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:49.091230 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:49.091187 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:49.297757 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:49.297721 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:49.699351 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:49.699312 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:49.933703 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:49.933667 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:50.395759 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:50.395733 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5d4pt" Jul 06 19:25:50.487122 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:50.487087 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:50.878446 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:50.878413 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:51.274343 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:51.274301 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:51.341954 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:51.341901 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:51.488117 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:51.488080 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:51.685596 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:51.685513 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:52.357829 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:52.357788 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:52.360396 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:52.360363 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/87921692-d802-479d-a041-a5ba5b5f78af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x6xzx\" (UID: \"87921692-d802-479d-a041-a5ba5b5f78af\") " pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:52.405060 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:52.405025 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f282m\"" Jul 06 19:25:52.412945 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:52.412925 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x6xzx" Jul 06 19:25:52.566086 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:52.566049 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x6xzx"] Jul 06 19:25:52.569729 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:52.569688 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87921692_d802_479d_a041_a5ba5b5f78af.slice/crio-5e5c2380760e648eba1814dbea76d3637b49dc47f37cd18d8e38790d96e4eed7 WatchSource:0}: Error finding container 5e5c2380760e648eba1814dbea76d3637b49dc47f37cd18d8e38790d96e4eed7: Status 404 returned error can't find the container with id 5e5c2380760e648eba1814dbea76d3637b49dc47f37cd18d8e38790d96e4eed7 Jul 06 19:25:53.560984 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:53.560941 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x6xzx" event={"ID":"87921692-d802-479d-a041-a5ba5b5f78af","Type":"ContainerStarted","Data":"5e5c2380760e648eba1814dbea76d3637b49dc47f37cd18d8e38790d96e4eed7"} Jul 06 19:25:55.792136 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:55.792097 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:55.792568 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:55.792167 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:55.795253 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:55.795227 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:25:55.796172 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:55.796151 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:25:55.805126 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:55.805094 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2cbb8d1-a657-43ca-a8d1-bc09a049a710-original-pull-secret\") pod \"global-pull-secret-syncer-cjr7k\" (UID: \"b2cbb8d1-a657-43ca-a8d1-bc09a049a710\") " pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:55.805269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:55.805147 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecb7b845-f496-49e7-87e3-a47752a5944b-metrics-certs\") pod \"network-metrics-daemon-5942g\" (UID: \"ecb7b845-f496-49e7-87e3-a47752a5944b\") " pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:55.889920 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:55.889861 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjr7k" Jul 06 19:25:55.893345 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:55.893309 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmz55\" (UniqueName: \"kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55\") pod \"network-check-target-ltflt\" (UID: \"8acd900e-9116-4692-a959-3ecf8035a515\") " pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:55.896264 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:55.896237 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmz55\" (UniqueName: \"kubernetes.io/projected/8acd900e-9116-4692-a959-3ecf8035a515-kube-api-access-hmz55\") pod \"network-check-target-ltflt\" (UID: \"8acd900e-9116-4692-a959-3ecf8035a515\") " pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:55.900505 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:55.900478 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qhtzw\"" Jul 06 19:25:55.908587 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:55.908561 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5942g" Jul 06 19:25:56.083795 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.083709 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jkxjr\"" Jul 06 19:25:56.091677 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.091652 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:56.452568 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.452469 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cjr7k"] Jul 06 19:25:56.457142 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:56.457112 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2cbb8d1_a657_43ca_a8d1_bc09a049a710.slice/crio-5e873f87e7cabea488ede4ea3d2545d25ec9596125e54070ddb95d58970cdb44 WatchSource:0}: Error finding container 5e873f87e7cabea488ede4ea3d2545d25ec9596125e54070ddb95d58970cdb44: Status 404 returned error can't find the container with id 5e873f87e7cabea488ede4ea3d2545d25ec9596125e54070ddb95d58970cdb44 Jul 06 19:25:56.472971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.472875 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5942g"] Jul 06 19:25:56.476748 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:56.476716 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecb7b845_f496_49e7_87e3_a47752a5944b.slice/crio-703c6a6cbeab2e9cd291e2e4b621d709d0e4fa312dc0b80a13f1d4d880fdf09d WatchSource:0}: Error finding container 703c6a6cbeab2e9cd291e2e4b621d709d0e4fa312dc0b80a13f1d4d880fdf09d: Status 404 returned error can't find the container with id 703c6a6cbeab2e9cd291e2e4b621d709d0e4fa312dc0b80a13f1d4d880fdf09d Jul 06 19:25:56.495249 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.495220 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ltflt"] Jul 06 19:25:56.498971 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:56.498938 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8acd900e_9116_4692_a959_3ecf8035a515.slice/crio-de7a1aac8bb57373744a60cc7c82f803c33d9fc2b64c4244ec21da5700ac125b WatchSource:0}: Error finding container de7a1aac8bb57373744a60cc7c82f803c33d9fc2b64c4244ec21da5700ac125b: Status 404 returned error can't find the container with id de7a1aac8bb57373744a60cc7c82f803c33d9fc2b64c4244ec21da5700ac125b Jul 06 19:25:56.577840 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.577801 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x6xzx" event={"ID":"87921692-d802-479d-a041-a5ba5b5f78af","Type":"ContainerStarted","Data":"8a30233e91c921a6d64cf6386ccaa708629b6e78d4db0d351eb9c1de83243963"} Jul 06 19:25:56.578977 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.578940 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cjr7k" event={"ID":"b2cbb8d1-a657-43ca-a8d1-bc09a049a710","Type":"ContainerStarted","Data":"5e873f87e7cabea488ede4ea3d2545d25ec9596125e54070ddb95d58970cdb44"} Jul 06 19:25:56.580252 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.580232 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ltflt" event={"ID":"8acd900e-9116-4692-a959-3ecf8035a515","Type":"ContainerStarted","Data":"245e6f2cfbea22395ca0da1b3a09e757b6e8d24a2132799f20f018c6adfa8d70"} Jul 06 19:25:56.580379 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.580267 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ltflt" event={"ID":"8acd900e-9116-4692-a959-3ecf8035a515","Type":"ContainerStarted","Data":"de7a1aac8bb57373744a60cc7c82f803c33d9fc2b64c4244ec21da5700ac125b"} Jul 06 19:25:56.580379 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.580317 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:25:56.581269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.581253 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5942g" event={"ID":"ecb7b845-f496-49e7-87e3-a47752a5944b","Type":"ContainerStarted","Data":"703c6a6cbeab2e9cd291e2e4b621d709d0e4fa312dc0b80a13f1d4d880fdf09d"} Jul 06 19:25:56.599569 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.599517 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ltflt" podStartSLOduration=66.599501291 podStartE2EDuration="1m6.599501291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:56.599182156 +0000 UTC m=+67.104467194" watchObservedRunningTime="2026-07-06 19:25:56.599501291 +0000 UTC m=+67.104786321" Jul 06 19:25:56.800488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.800452 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:56.800488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.800494 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:56.801106 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.800521 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:56.801106 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.800539 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:56.801106 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.800563 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:56.801106 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.800613 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:56.801340 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.801315 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2369af8d-444e-488a-97c3-692ee5138a01-service-ca-bundle\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:56.803076 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.803051 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/55445c8d-30b3-47cd-9479-1ca223c726e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-lx4sd\" (UID: \"55445c8d-30b3-47cd-9479-1ca223c726e5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:56.803228 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.803206 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1e15d9b3-599b-4f3b-8571-12b7a36a6d3e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-nmz4s\" (UID: \"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:56.803274 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.803206 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls\") pod \"image-registry-74b85d8d69-gq2gp\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:56.803435 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.803416 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/752bf161-7c8b-4af8-9d5d-3a7f9260e693-metrics-tls\") pod \"dns-default-gd48l\" (UID: \"752bf161-7c8b-4af8-9d5d-3a7f9260e693\") " pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:56.803498 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.803483 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2369af8d-444e-488a-97c3-692ee5138a01-metrics-certs\") pod \"router-default-756fc79d99-fvh2j\" (UID: \"2369af8d-444e-488a-97c3-692ee5138a01\") " pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:56.931601 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.931571 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-plcs2\"" Jul 06 19:25:56.939763 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.939726 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:56.978548 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.978515 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-s7t8b\"" Jul 06 19:25:56.986733 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.986452 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" Jul 06 19:25:56.997689 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:56.997611 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2ltbj\"" Jul 06 19:25:57.005353 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.005124 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" Jul 06 19:25:57.016357 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.016148 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-khkrm\"" Jul 06 19:25:57.024416 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.024384 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gd48l" Jul 06 19:25:57.029427 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.029200 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zp7lr\"" Jul 06 19:25:57.037575 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.037526 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:57.107492 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.107444 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b85d8d69-gq2gp"] Jul 06 19:25:57.127319 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:57.127059 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd7bc946_cbb8_4966_b8c5_6edb16cc366e.slice/crio-3acbb4bdd3453d1725219c97b6f9f3b71b6a0f0a3ce4762b34a6b68342f4d5f5 WatchSource:0}: Error finding container 3acbb4bdd3453d1725219c97b6f9f3b71b6a0f0a3ce4762b34a6b68342f4d5f5: Status 404 returned error can't find the container with id 3acbb4bdd3453d1725219c97b6f9f3b71b6a0f0a3ce4762b34a6b68342f4d5f5 Jul 06 19:25:57.320061 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.319835 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gd48l"] Jul 06 19:25:57.337149 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.337068 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s"] Jul 06 19:25:57.360041 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.359967 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd"] Jul 06 19:25:57.364695 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:57.364663 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55445c8d_30b3_47cd_9479_1ca223c726e5.slice/crio-a16596613f86b65d82d233d918a283f33f05dd2847a232d3f5aabb15a5a791ac WatchSource:0}: Error finding container a16596613f86b65d82d233d918a283f33f05dd2847a232d3f5aabb15a5a791ac: Status 404 returned error can't find the container with id a16596613f86b65d82d233d918a283f33f05dd2847a232d3f5aabb15a5a791ac Jul 06 19:25:57.398877 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.398778 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-756fc79d99-fvh2j"] Jul 06 19:25:57.404146 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:25:57.404115 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2369af8d_444e_488a_97c3_692ee5138a01.slice/crio-96f6133a1fd817f2ac6338ad9af3a675c16247c56d863dcd34e0c2706f6ce392 WatchSource:0}: Error finding container 96f6133a1fd817f2ac6338ad9af3a675c16247c56d863dcd34e0c2706f6ce392: Status 404 returned error can't find the container with id 96f6133a1fd817f2ac6338ad9af3a675c16247c56d863dcd34e0c2706f6ce392 Jul 06 19:25:57.591798 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.591707 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gd48l" event={"ID":"752bf161-7c8b-4af8-9d5d-3a7f9260e693","Type":"ContainerStarted","Data":"126295cb38843f6ae182d624421ba7feb9d2bb8fd57097444c96b5beb7842231"} Jul 06 19:25:57.593177 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.593136 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" event={"ID":"55445c8d-30b3-47cd-9479-1ca223c726e5","Type":"ContainerStarted","Data":"a16596613f86b65d82d233d918a283f33f05dd2847a232d3f5aabb15a5a791ac"} Jul 06 19:25:57.595283 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.595250 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-756fc79d99-fvh2j" event={"ID":"2369af8d-444e-488a-97c3-692ee5138a01","Type":"ContainerStarted","Data":"f1764613ed030cd49e132b48d0e05c08b285a4091739b0eeec3cc1e0f310ea18"} Jul 06 19:25:57.595415 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.595310 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-756fc79d99-fvh2j" event={"ID":"2369af8d-444e-488a-97c3-692ee5138a01","Type":"ContainerStarted","Data":"96f6133a1fd817f2ac6338ad9af3a675c16247c56d863dcd34e0c2706f6ce392"} Jul 06 19:25:57.597808 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.597708 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x6xzx" event={"ID":"87921692-d802-479d-a041-a5ba5b5f78af","Type":"ContainerStarted","Data":"69fa65d515325c9254a6e6f6953c15ba8fbab4e23de5920ebabe19e62dc0915c"} Jul 06 19:25:57.599596 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.599506 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" event={"ID":"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e","Type":"ContainerStarted","Data":"42b0b1f3baf3b801edbd451db4e44efd96b6f19b281bde2cd536c10a89e18c9e"} Jul 06 19:25:57.601866 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.601839 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" event={"ID":"cd7bc946-cbb8-4966-b8c5-6edb16cc366e","Type":"ContainerStarted","Data":"e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f"} Jul 06 19:25:57.602007 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.601884 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" event={"ID":"cd7bc946-cbb8-4966-b8c5-6edb16cc366e","Type":"ContainerStarted","Data":"3acbb4bdd3453d1725219c97b6f9f3b71b6a0f0a3ce4762b34a6b68342f4d5f5"} Jul 06 19:25:57.617627 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.617505 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-756fc79d99-fvh2j" podStartSLOduration=43.617483303 podStartE2EDuration="43.617483303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:57.616863804 +0000 UTC m=+68.122148844" watchObservedRunningTime="2026-07-06 19:25:57.617483303 +0000 UTC m=+68.122768340" Jul 06 19:25:57.640545 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:57.639998 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" podStartSLOduration=67.639974366 podStartE2EDuration="1m7.639974366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:57.638169688 +0000 UTC m=+68.143454765" watchObservedRunningTime="2026-07-06 19:25:57.639974366 +0000 UTC m=+68.145259404" Jul 06 19:25:58.039003 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.038811 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:58.046713 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.046682 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:58.612479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.612348 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5942g" event={"ID":"ecb7b845-f496-49e7-87e3-a47752a5944b","Type":"ContainerStarted","Data":"bbd2a7325320f276e6358ffd38605c732e8e43faba2a1a94bd6fe1e457a4a552"} Jul 06 19:25:58.612479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.612412 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:58.612479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.612438 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5942g" event={"ID":"ecb7b845-f496-49e7-87e3-a47752a5944b","Type":"ContainerStarted","Data":"04b1862309fec72cfcbe286fcb65cff607b90ed01106b437045d9821384f2de7"} Jul 06 19:25:58.613290 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.612726 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:25:58.613971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.613896 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-756fc79d99-fvh2j" Jul 06 19:25:58.676765 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.676177 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5942g" podStartSLOduration=67.375985766 podStartE2EDuration="1m8.676153183s" totalImagesPullingTime="1.300167417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:58.640821302 +0000 UTC m=+69.146106332" watchObservedRunningTime="2026-07-06 19:25:58.676153183 +0000 UTC m=+69.181438222" Jul 06 19:25:58.709381 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.709343 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh"] Jul 06 19:25:58.727582 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.727245 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv"] Jul 06 19:25:58.728271 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.727881 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:25:58.734088 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.734065 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t"] Jul 06 19:25:58.735136 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.735113 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 19:25:58.735436 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.735419 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 19:25:58.735648 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.735630 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 19:25:58.735872 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.735854 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 19:25:58.736250 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.736227 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.738143 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.737710 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t" Jul 06 19:25:58.741936 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.741891 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 06 19:25:58.742035 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.741973 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh"] Jul 06 19:25:58.742035 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.742013 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv"] Jul 06 19:25:58.742035 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.742034 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t"] Jul 06 19:25:58.742219 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.742200 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 06 19:25:58.742369 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.742351 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 06 19:25:58.742567 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.742523 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 06 19:25:58.742713 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.742457 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 19:25:58.742895 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.742875 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-25hfs\"" Jul 06 19:25:58.824948 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.824697 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwpkk\" (UniqueName: \"kubernetes.io/projected/1da92b45-ff30-4a3f-a418-75339c7f1cbb-kube-api-access-zwpkk\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.824948 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.824792 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.824948 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.824822 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.824948 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.824850 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/724b5bae-2e94-46be-8c70-9a7b5080b628-tmp\") pod \"klusterlet-addon-workmgr-794fbcd44f-4f8jh\" (UID: \"724b5bae-2e94-46be-8c70-9a7b5080b628\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:25:58.824948 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.824875 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-hub\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.824948 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.824899 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da92b45-ff30-4a3f-a418-75339c7f1cbb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.825393 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.824967 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-924ct\" (UniqueName: \"kubernetes.io/projected/724b5bae-2e94-46be-8c70-9a7b5080b628-kube-api-access-924ct\") pod \"klusterlet-addon-workmgr-794fbcd44f-4f8jh\" (UID: \"724b5bae-2e94-46be-8c70-9a7b5080b628\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:25:58.825393 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.824992 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-ca\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.825393 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.825018 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/724b5bae-2e94-46be-8c70-9a7b5080b628-klusterlet-config\") pod \"klusterlet-addon-workmgr-794fbcd44f-4f8jh\" (UID: \"724b5bae-2e94-46be-8c70-9a7b5080b628\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:25:58.883570 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.883538 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:58.926971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.925675 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.926971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.925723 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.926971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.925757 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/724b5bae-2e94-46be-8c70-9a7b5080b628-tmp\") pod \"klusterlet-addon-workmgr-794fbcd44f-4f8jh\" (UID: \"724b5bae-2e94-46be-8c70-9a7b5080b628\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:25:58.926971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.925786 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-hub\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.926971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.925811 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da92b45-ff30-4a3f-a418-75339c7f1cbb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.926971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.925865 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78489d56-p885t\" (UID: \"0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t" Jul 06 19:25:58.926971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.925924 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-924ct\" (UniqueName: \"kubernetes.io/projected/724b5bae-2e94-46be-8c70-9a7b5080b628-kube-api-access-924ct\") pod \"klusterlet-addon-workmgr-794fbcd44f-4f8jh\" (UID: \"724b5bae-2e94-46be-8c70-9a7b5080b628\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:25:58.926971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.925951 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-ca\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.926971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.925977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/724b5bae-2e94-46be-8c70-9a7b5080b628-klusterlet-config\") pod \"klusterlet-addon-workmgr-794fbcd44f-4f8jh\" (UID: \"724b5bae-2e94-46be-8c70-9a7b5080b628\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:25:58.926971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.926047 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwpkk\" (UniqueName: \"kubernetes.io/projected/1da92b45-ff30-4a3f-a418-75339c7f1cbb-kube-api-access-zwpkk\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.926971 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.926120 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28dnh\" (UniqueName: \"kubernetes.io/projected/0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91-kube-api-access-28dnh\") pod \"managed-serviceaccount-addon-agent-78489d56-p885t\" (UID: \"0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t" Jul 06 19:25:58.929727 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.928732 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/724b5bae-2e94-46be-8c70-9a7b5080b628-tmp\") pod \"klusterlet-addon-workmgr-794fbcd44f-4f8jh\" (UID: \"724b5bae-2e94-46be-8c70-9a7b5080b628\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:25:58.929727 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.929518 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da92b45-ff30-4a3f-a418-75339c7f1cbb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.930820 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.930790 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.931023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.931001 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.940828 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.940798 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-hub\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.943133 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.943061 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/724b5bae-2e94-46be-8c70-9a7b5080b628-klusterlet-config\") pod \"klusterlet-addon-workmgr-794fbcd44f-4f8jh\" (UID: \"724b5bae-2e94-46be-8c70-9a7b5080b628\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:25:58.944017 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.943972 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1da92b45-ff30-4a3f-a418-75339c7f1cbb-ca\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.954336 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.954274 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwpkk\" (UniqueName: \"kubernetes.io/projected/1da92b45-ff30-4a3f-a418-75339c7f1cbb-kube-api-access-zwpkk\") pod \"cluster-proxy-proxy-agent-7d7f544c8f-qwqvv\" (UID: \"1da92b45-ff30-4a3f-a418-75339c7f1cbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:58.955870 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.955812 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-924ct\" (UniqueName: \"kubernetes.io/projected/724b5bae-2e94-46be-8c70-9a7b5080b628-kube-api-access-924ct\") pod \"klusterlet-addon-workmgr-794fbcd44f-4f8jh\" (UID: \"724b5bae-2e94-46be-8c70-9a7b5080b628\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:25:58.959032 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:58.958961 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:59.028079 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.026855 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78489d56-p885t\" (UID: \"0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t" Jul 06 19:25:59.028079 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.027031 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28dnh\" (UniqueName: \"kubernetes.io/projected/0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91-kube-api-access-28dnh\") pod \"managed-serviceaccount-addon-agent-78489d56-p885t\" (UID: \"0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t" Jul 06 19:25:59.031045 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.030982 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78489d56-p885t\" (UID: \"0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t" Jul 06 19:25:59.040447 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.040364 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28dnh\" (UniqueName: \"kubernetes.io/projected/0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91-kube-api-access-28dnh\") pod \"managed-serviceaccount-addon-agent-78489d56-p885t\" (UID: \"0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t" Jul 06 19:25:59.051978 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.051941 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:25:59.058831 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.058750 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:59.073382 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.073115 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" Jul 06 19:25:59.082134 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.080743 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t" Jul 06 19:25:59.158293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.158214 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:59.267900 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.267806 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:59.748807 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.748768 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:59.828561 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:25:59.828517 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:00.172248 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:00.172218 2572 scope.go:117] "RemoveContainer" containerID="ee60c6941ffd979dceab6f834e524fe1d41da0cc5f8748d39d5e0e330f2199a6" Jul 06 19:26:02.361899 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:02.361858 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:26:02.364862 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:02.364828 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/94235910-e2b4-4f48-9c04-9a01df47b39e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fqdb4\" (UID: \"94235910-e2b4-4f48-9c04-9a01df47b39e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:26:02.581566 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:02.581528 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2fj6l\"" Jul 06 19:26:02.589568 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:02.589536 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" Jul 06 19:26:03.426026 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.420977 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t"] Jul 06 19:26:03.536524 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.536456 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv"] Jul 06 19:26:03.548146 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:26:03.547990 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1da92b45_ff30_4a3f_a418_75339c7f1cbb.slice/crio-ff90c0015cdf78ecbde2965a3a29aae6f04eb2b7e16ee574c920d7373748ce03 WatchSource:0}: Error finding container ff90c0015cdf78ecbde2965a3a29aae6f04eb2b7e16ee574c920d7373748ce03: Status 404 returned error can't find the container with id ff90c0015cdf78ecbde2965a3a29aae6f04eb2b7e16ee574c920d7373748ce03 Jul 06 19:26:03.575205 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.574793 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh"] Jul 06 19:26:03.598611 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.596130 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4"] Jul 06 19:26:03.609512 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:26:03.609472 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94235910_e2b4_4f48_9c04_9a01df47b39e.slice/crio-bf03b542f2b30dcd09fccd50c2fe8e3b596bbfdb1df1aa2328bcb1c2afbb4035 WatchSource:0}: Error finding container bf03b542f2b30dcd09fccd50c2fe8e3b596bbfdb1df1aa2328bcb1c2afbb4035: Status 404 returned error can't find the container with id bf03b542f2b30dcd09fccd50c2fe8e3b596bbfdb1df1aa2328bcb1c2afbb4035 Jul 06 19:26:03.643135 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.643097 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gd48l" event={"ID":"752bf161-7c8b-4af8-9d5d-3a7f9260e693","Type":"ContainerStarted","Data":"ad199c04c35f9124873c8d85bf5109f8e67a0b48a3d1c66a64d05629ebb33458"} Jul 06 19:26:03.650952 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.649947 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" event={"ID":"1da92b45-ff30-4a3f-a418-75339c7f1cbb","Type":"ContainerStarted","Data":"ff90c0015cdf78ecbde2965a3a29aae6f04eb2b7e16ee574c920d7373748ce03"} Jul 06 19:26:03.656413 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.655650 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" event={"ID":"55445c8d-30b3-47cd-9479-1ca223c726e5","Type":"ContainerStarted","Data":"30bd7e0d5fa4ae561c8b199374684fd14367ba773fc228ddaf89d6ecc2660e94"} Jul 06 19:26:03.661468 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.661439 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t" event={"ID":"0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91","Type":"ContainerStarted","Data":"86f26791c1f0b9ac00dabe10514e0a1f4a1ca289f30963f9945f8edd0013b6af"} Jul 06 19:26:03.670046 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.669975 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x6xzx" event={"ID":"87921692-d802-479d-a041-a5ba5b5f78af","Type":"ContainerStarted","Data":"a17dd79517a6964c7e9c1d2ff55ec276a456d7ffa4e5f8b6905efae6f27e06f7"} Jul 06 19:26:03.672318 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.672247 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" event={"ID":"94235910-e2b4-4f48-9c04-9a01df47b39e","Type":"ContainerStarted","Data":"bf03b542f2b30dcd09fccd50c2fe8e3b596bbfdb1df1aa2328bcb1c2afbb4035"} Jul 06 19:26:03.678424 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.678398 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" event={"ID":"724b5bae-2e94-46be-8c70-9a7b5080b628","Type":"ContainerStarted","Data":"f5431649737ed45b88e215af4aa2e6b10de76fa898cc584cd0ea34c4d612c18d"} Jul 06 19:26:03.680390 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.680370 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" event={"ID":"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e","Type":"ContainerStarted","Data":"7748861303ce931ec0e2ac33259b03b88a5c29396825b1793b03daa95252bc68"} Jul 06 19:26:03.683510 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.683465 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:26:03.683612 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.683544 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" event={"ID":"5bd27dda-064e-48ba-8ae7-d5a14b609bd3","Type":"ContainerStarted","Data":"64129c5c5df0d4cf6f2847a310442063642b17ea4ab33a33b9872c716e0b4830"} Jul 06 19:26:03.684401 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.684329 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:26:03.688549 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.688060 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cjr7k" event={"ID":"b2cbb8d1-a657-43ca-a8d1-bc09a049a710","Type":"ContainerStarted","Data":"1c18b6ddf760466e2b7c42944ed780f6a5e08ab9a7fe4c91dc609d896f23f352"} Jul 06 19:26:03.722552 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.722488 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-lx4sd" podStartSLOduration=43.786427122 podStartE2EDuration="49.72246399s" totalImagesPullingTime="5.936036868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:03.696379327 +0000 UTC m=+74.201664366" watchObservedRunningTime="2026-07-06 19:26:03.72246399 +0000 UTC m=+74.227749030" Jul 06 19:26:03.723452 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.723121 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" podStartSLOduration=40.371664503 podStartE2EDuration="49.723104416s" totalImagesPullingTime="9.351439913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:03.723038233 +0000 UTC m=+74.228323271" watchObservedRunningTime="2026-07-06 19:26:03.723104416 +0000 UTC m=+74.228389454" Jul 06 19:26:03.742017 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.741082 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cjr7k" podStartSLOduration=66.875269927 podStartE2EDuration="1m13.741059585s" totalImagesPullingTime="6.865789658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:03.741046595 +0000 UTC m=+74.246331635" watchObservedRunningTime="2026-07-06 19:26:03.741059585 +0000 UTC m=+74.246344624" Jul 06 19:26:03.767475 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.767087 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x6xzx" podStartSLOduration=17.25152286 podStartE2EDuration="27.767067517s" totalImagesPullingTime="10.515544657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:25:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:03.766331713 +0000 UTC m=+74.271616752" watchObservedRunningTime="2026-07-06 19:26:03.767067517 +0000 UTC m=+74.272352534" Jul 06 19:26:03.845624 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.845572 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:03.948211 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:03.948169 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:04.051300 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:04.051262 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:04.686791 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:04.686746 2572 patch_prober.go:28] interesting pod/console-operator-68b757fb67-6cqbs container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.10:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 19:26:04.687240 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:04.686823 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" podUID="5bd27dda-064e-48ba-8ae7-d5a14b609bd3" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.10:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 06 19:26:04.696963 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:04.696925 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" event={"ID":"1e15d9b3-599b-4f3b-8571-12b7a36a6d3e","Type":"ContainerStarted","Data":"de60ea880c0bc651c619717c4b53b5309f25e0dae196c578a96e379a18fe07f6"} Jul 06 19:26:04.704215 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:04.703270 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gd48l" event={"ID":"752bf161-7c8b-4af8-9d5d-3a7f9260e693","Type":"ContainerStarted","Data":"0008d01a00e560c505cde035d699109fde30398b54058b4aa90f326e2a2ed2e8"} Jul 06 19:26:04.704215 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:04.703318 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gd48l" Jul 06 19:26:04.710668 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:04.710614 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:04.716164 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:04.716103 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-nmz4s" podStartSLOduration=44.827807317 podStartE2EDuration="50.716079756s" totalImagesPullingTime="5.888272439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:04.715411252 +0000 UTC m=+75.220696289" watchObservedRunningTime="2026-07-06 19:26:04.716079756 +0000 UTC m=+75.221364795" Jul 06 19:26:05.072102 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:05.072050 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:05.166389 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:05.166337 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:05.496272 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:05.496007 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-6cqbs" Jul 06 19:26:05.533401 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:05.533028 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gd48l" podStartSLOduration=35.614227406 podStartE2EDuration="41.533004025s" totalImagesPullingTime="5.918776619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:04.743203994 +0000 UTC m=+75.248489033" watchObservedRunningTime="2026-07-06 19:26:05.533004025 +0000 UTC m=+76.038289065" Jul 06 19:26:05.850442 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:05.850348 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:05.942403 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:05.942362 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:06.029401 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:06.029347 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:06.366496 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:06.366463 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:06.433411 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:06.433375 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:06.771326 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:06.771277 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:06.873055 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:06.872944 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:07.416209 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:07.416169 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:07.515720 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:07.515679 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:07.616325 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:07.616288 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:07.716898 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:07.716859 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:07.721347 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:07.721312 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" event={"ID":"94235910-e2b4-4f48-9c04-9a01df47b39e","Type":"ContainerStarted","Data":"72ece5bd6a8bf3de6ed9accd91cec007813132a94ab16b86f309b1ddaf907e8c"} Jul 06 19:26:07.740454 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:07.740382 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fqdb4" podStartSLOduration=34.185259333 podStartE2EDuration="37.7403603s" totalImagesPullingTime="3.555100967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:07.73976926 +0000 UTC m=+78.245054294" watchObservedRunningTime="2026-07-06 19:26:07.7403603 +0000 UTC m=+78.245645338" Jul 06 19:26:07.926434 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:07.926339 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:07.997652 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:07.997607 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:08.671253 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:08.671206 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:08.728177 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:08.728142 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t" event={"ID":"0b249e0d-c0b4-4b4a-9a4a-1bbbdbb86b91","Type":"ContainerStarted","Data":"976b1cb26f958435316f528fc4c02e9602d526f33c88212a4478e5d976eb7ca2"} Jul 06 19:26:08.736633 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:08.736584 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:08.754816 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:08.754581 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78489d56-p885t" podStartSLOduration=6.565938739 podStartE2EDuration="10.754558801s" totalImagesPullingTime="4.188620062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:08.753457747 +0000 UTC m=+79.258742784" watchObservedRunningTime="2026-07-06 19:26:08.754558801 +0000 UTC m=+79.259843842" Jul 06 19:26:09.122598 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:09.122565 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:09.197372 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:09.197270 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:09.309881 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:09.309829 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:11.597206 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:11.597166 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:11.662262 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:11.662171 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:11.743825 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:11.743789 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" event={"ID":"1da92b45-ff30-4a3f-a418-75339c7f1cbb","Type":"ContainerStarted","Data":"fe4c9f506312f1be1a004593a76fbe7abcf189203f1ce85f98a6c07e52bd853d"} Jul 06 19:26:11.744026 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:11.743842 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" event={"ID":"1da92b45-ff30-4a3f-a418-75339c7f1cbb","Type":"ContainerStarted","Data":"6557e67818fefc004fed717b6f9daa20234d99e39ceed2d0bdb85f7e8b99b3b7"} Jul 06 19:26:11.744026 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:11.743865 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" event={"ID":"1da92b45-ff30-4a3f-a418-75339c7f1cbb","Type":"ContainerStarted","Data":"34e7349a222343e832ab03e5da234ea5cda90a2f8cb6eb35980481cc35d7e015"} Jul 06 19:26:11.745367 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:11.745341 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" event={"ID":"724b5bae-2e94-46be-8c70-9a7b5080b628","Type":"ContainerStarted","Data":"47a99ac33e405e1d95741fff694835a4cff565bb81767be29d9cf7067ef47985"} Jul 06 19:26:11.745565 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:11.745544 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:26:11.747216 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:11.747195 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" Jul 06 19:26:11.769920 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:11.769853 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d7f544c8f-qwqvv" podStartSLOduration=6.17853949 podStartE2EDuration="13.769836306s" totalImagesPullingTime="7.591296816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:11.768533669 +0000 UTC m=+82.273818726" watchObservedRunningTime="2026-07-06 19:26:11.769836306 +0000 UTC m=+82.275121343" Jul 06 19:26:11.785245 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:11.785185 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794fbcd44f-4f8jh" podStartSLOduration=6.237997397 podStartE2EDuration="13.785165146s" totalImagesPullingTime="7.547167749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:11.783681008 +0000 UTC m=+82.288966048" watchObservedRunningTime="2026-07-06 19:26:11.785165146 +0000 UTC m=+82.290450184" Jul 06 19:26:13.030011 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:13.029884 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:13.258230 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:13.258197 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:13.359622 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:13.359525 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:13.429325 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:13.429290 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:13.850509 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:13.850473 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:13.939545 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:13.939499 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:14.712161 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:14.712128 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gd48l" Jul 06 19:26:15.450316 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:15.450266 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:15.551276 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:15.551245 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:16.224826 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.224776 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:16.365519 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.365484 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c2h4w"] Jul 06 19:26:16.369061 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.369042 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.373599 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.373573 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:16.374177 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.374156 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 19:26:16.375341 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.375322 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 19:26:16.375551 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.375537 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 19:26:16.377719 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.377694 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rm8wk\"" Jul 06 19:26:16.378131 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.378110 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 19:26:16.379470 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.379447 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-textfile\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.379557 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.379479 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-tls\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.379557 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.379506 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-accelerators-collector-config\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.379693 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.379566 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8beb7b2f-4c3e-493c-b734-c75fadc161ad-metrics-client-ca\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.379693 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.379599 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtptc\" (UniqueName: \"kubernetes.io/projected/8beb7b2f-4c3e-493c-b734-c75fadc161ad-kube-api-access-wtptc\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.379693 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.379641 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-wtmp\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.379793 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.379704 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8beb7b2f-4c3e-493c-b734-c75fadc161ad-sys\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.379793 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.379772 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8beb7b2f-4c3e-493c-b734-c75fadc161ad-root\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.379859 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.379815 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.480820 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.480723 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtptc\" (UniqueName: \"kubernetes.io/projected/8beb7b2f-4c3e-493c-b734-c75fadc161ad-kube-api-access-wtptc\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.480820 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.480777 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-wtmp\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.480896 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-wtmp\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.480898 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8beb7b2f-4c3e-493c-b734-c75fadc161ad-sys\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.480972 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8beb7b2f-4c3e-493c-b734-c75fadc161ad-root\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.480983 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8beb7b2f-4c3e-493c-b734-c75fadc161ad-sys\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.481008 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.481030 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8beb7b2f-4c3e-493c-b734-c75fadc161ad-root\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.481035 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-textfile\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.481056 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-tls\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.481075 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-accelerators-collector-config\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.481095 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8beb7b2f-4c3e-493c-b734-c75fadc161ad-metrics-client-ca\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481521 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.481374 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-textfile\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481697 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.481677 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8beb7b2f-4c3e-493c-b734-c75fadc161ad-metrics-client-ca\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.481804 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.481783 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-accelerators-collector-config\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.483670 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.483646 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-tls\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.483808 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.483788 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8beb7b2f-4c3e-493c-b734-c75fadc161ad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.493513 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.493485 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtptc\" (UniqueName: \"kubernetes.io/projected/8beb7b2f-4c3e-493c-b734-c75fadc161ad-kube-api-access-wtptc\") pod \"node-exporter-c2h4w\" (UID: \"8beb7b2f-4c3e-493c-b734-c75fadc161ad\") " pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.496965 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.496937 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:16.624631 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.624601 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:16.679605 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.679569 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c2h4w" Jul 06 19:26:16.691511 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:26:16.691483 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8beb7b2f_4c3e_493c_b734_c75fadc161ad.slice/crio-09a51c560a5c100947dbde882512530b0b6ab81e740100c4a7260207166320b9 WatchSource:0}: Error finding container 09a51c560a5c100947dbde882512530b0b6ab81e740100c4a7260207166320b9: Status 404 returned error can't find the container with id 09a51c560a5c100947dbde882512530b0b6ab81e740100c4a7260207166320b9 Jul 06 19:26:16.767027 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.766940 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2h4w" event={"ID":"8beb7b2f-4c3e-493c-b734-c75fadc161ad","Type":"ContainerStarted","Data":"09a51c560a5c100947dbde882512530b0b6ab81e740100c4a7260207166320b9"} Jul 06 19:26:16.944251 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.944210 2572 patch_prober.go:28] interesting pod/image-registry-74b85d8d69-gq2gp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 19:26:16.944509 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:16.944277 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" podUID="cd7bc946-cbb8-4966-b8c5-6edb16cc366e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:26:17.035429 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:17.035329 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:17.130865 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:17.130831 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:17.231662 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:17.231624 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:17.370102 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:17.369861 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:17.650202 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:17.650113 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:18.777502 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:18.777467 2572 generic.go:356] "Generic (PLEG): container finished" podID="8beb7b2f-4c3e-493c-b734-c75fadc161ad" containerID="0d589ea8c816533b553777bf09171440b49e3c5b0982cded0ce240a20d07d43d" exitCode=0 Jul 06 19:26:18.778022 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:18.777535 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2h4w" event={"ID":"8beb7b2f-4c3e-493c-b734-c75fadc161ad","Type":"ContainerDied","Data":"0d589ea8c816533b553777bf09171440b49e3c5b0982cded0ce240a20d07d43d"} Jul 06 19:26:19.620984 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:19.620952 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:26:19.729588 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:19.729551 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:19.784540 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:19.784508 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2h4w" event={"ID":"8beb7b2f-4c3e-493c-b734-c75fadc161ad","Type":"ContainerStarted","Data":"37b3afe3cda21358a049f6658898b363d91f361f04e99971912ab756e03318c1"} Jul 06 19:26:19.785052 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:19.784605 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2h4w" event={"ID":"8beb7b2f-4c3e-493c-b734-c75fadc161ad","Type":"ContainerStarted","Data":"b4ae9f996ef94f83febfa5900150614bb40b64c1401c03c3d9c9c45678b7b148"} Jul 06 19:26:19.816225 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:19.816165 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c2h4w" podStartSLOduration=2.686139832 podStartE2EDuration="3.816149271s" totalImagesPullingTime="1.056609763s" totalInitContainerRuntime="73.399676ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:19.815065233 +0000 UTC m=+90.320350296" watchObservedRunningTime="2026-07-06 19:26:19.816149271 +0000 UTC m=+90.321434308" Jul 06 19:26:19.830868 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:19.830840 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:19.894602 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:19.894505 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:20.377245 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:20.377211 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:20.455389 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:20.455349 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:20.765833 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:20.765782 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:21.538626 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:21.538584 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74b85d8d69-gq2gp"] Jul 06 19:26:21.644366 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:21.644330 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:21.748349 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:21.748314 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:21.809241 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:21.809146 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:22.661811 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:22.661773 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:22.741973 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:22.741928 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:23.926698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:23.926651 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:23.999947 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:23.999901 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:26.532606 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:26.532574 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:26.594960 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:26.594900 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:27.012608 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:27.012574 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:27.101450 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:27.101407 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:27.604256 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:27.604221 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ltflt" Jul 06 19:26:27.635114 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:27.635077 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:27.733574 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:27.733539 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:28.236875 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:28.236821 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:28.312511 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:28.312475 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:30.511643 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:30.511610 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:30.719176 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:30.719138 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:30.818410 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:30.818326 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:32.612128 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:32.612093 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:32.700829 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:32.700777 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:33.005924 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:33.005868 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:33.115475 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:33.115427 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:34.693967 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:34.693894 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:34.802770 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:34.802725 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:36.588553 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:36.588516 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:36.659969 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:36.659934 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:36.725565 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:36.725530 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:41.531158 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:41.531124 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:41.597251 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:41.597213 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:41.871602 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:41.871520 2572 generic.go:356] "Generic (PLEG): container finished" podID="239c95d1-5bdb-4078-ad6b-9eb593957bb6" containerID="ef2403078f58ff33f1c114f029aacc1eec88b8d335573e0424bc8fbe4d9f0d52" exitCode=0 Jul 06 19:26:41.871602 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:41.871589 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" event={"ID":"239c95d1-5bdb-4078-ad6b-9eb593957bb6","Type":"ContainerDied","Data":"ef2403078f58ff33f1c114f029aacc1eec88b8d335573e0424bc8fbe4d9f0d52"} Jul 06 19:26:41.871951 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:41.871937 2572 scope.go:117] "RemoveContainer" containerID="ef2403078f58ff33f1c114f029aacc1eec88b8d335573e0424bc8fbe4d9f0d52" Jul 06 19:26:42.879716 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:42.878978 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rg9vj" event={"ID":"239c95d1-5bdb-4078-ad6b-9eb593957bb6","Type":"ContainerStarted","Data":"4d7f861b79aa0d295ddb752809e876f22983bf3cc15da8b3dbda19c6a4c98afb"} Jul 06 19:26:46.560098 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.560055 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" podUID="cd7bc946-cbb8-4966-b8c5-6edb16cc366e" containerName="registry" containerID="cri-o://e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f" gracePeriod=30 Jul 06 19:26:46.855861 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.855830 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:26:46.857181 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.857157 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:46.895417 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.895365 2572 generic.go:356] "Generic (PLEG): container finished" podID="cd7bc946-cbb8-4966-b8c5-6edb16cc366e" containerID="e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f" exitCode=0 Jul 06 19:26:46.895417 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.895415 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" event={"ID":"cd7bc946-cbb8-4966-b8c5-6edb16cc366e","Type":"ContainerDied","Data":"e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f"} Jul 06 19:26:46.895728 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.895433 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" Jul 06 19:26:46.895728 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.895460 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b85d8d69-gq2gp" event={"ID":"cd7bc946-cbb8-4966-b8c5-6edb16cc366e","Type":"ContainerDied","Data":"3acbb4bdd3453d1725219c97b6f9f3b71b6a0f0a3ce4762b34a6b68342f4d5f5"} Jul 06 19:26:46.895728 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.895487 2572 scope.go:117] "RemoveContainer" containerID="e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f" Jul 06 19:26:46.905060 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.905036 2572 scope.go:117] "RemoveContainer" containerID="e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f" Jul 06 19:26:46.905438 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:26:46.905409 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f\": container with ID starting with e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f not found: ID does not exist" containerID="e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f" Jul 06 19:26:46.905536 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.905447 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f"} err="failed to get container status \"e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f\": rpc error: code = NotFound desc = could not find container \"e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f\": container with ID starting with e085e3d6e01136052e3179807257e1c58c7a711fbc8ba2e9fc8d90daf63c003f not found: ID does not exist" Jul 06 19:26:46.938561 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.938528 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-trusted-ca\") pod \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " Jul 06 19:26:46.938561 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.938576 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-certificates\") pod \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " Jul 06 19:26:46.938821 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.938635 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-ca-trust-extracted\") pod \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " Jul 06 19:26:46.938821 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.938663 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-installation-pull-secrets\") pod \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " Jul 06 19:26:46.938821 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.938702 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls\") pod \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " Jul 06 19:26:46.938821 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.938771 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-bound-sa-token\") pod \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " Jul 06 19:26:46.938821 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.938799 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7cr7\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-kube-api-access-j7cr7\") pod \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " Jul 06 19:26:46.939074 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.939029 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-image-registry-private-configuration\") pod \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\" (UID: \"cd7bc946-cbb8-4966-b8c5-6edb16cc366e\") " Jul 06 19:26:46.939074 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.939050 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cd7bc946-cbb8-4966-b8c5-6edb16cc366e" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:26:46.939178 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.939090 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cd7bc946-cbb8-4966-b8c5-6edb16cc366e" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:26:46.939334 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.939308 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-trusted-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:26:46.939428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.939336 2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-certificates\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:26:46.941424 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.941388 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cd7bc946-cbb8-4966-b8c5-6edb16cc366e" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:26:46.941560 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.941513 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-kube-api-access-j7cr7" (OuterVolumeSpecName: "kube-api-access-j7cr7") pod "cd7bc946-cbb8-4966-b8c5-6edb16cc366e" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e"). InnerVolumeSpecName "kube-api-access-j7cr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:26:46.941560 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.941521 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cd7bc946-cbb8-4966-b8c5-6edb16cc366e" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:26:46.941721 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.941691 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cd7bc946-cbb8-4966-b8c5-6edb16cc366e" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:26:46.941822 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.941775 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cd7bc946-cbb8-4966-b8c5-6edb16cc366e" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:26:46.947753 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.947728 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cd7bc946-cbb8-4966-b8c5-6edb16cc366e" (UID: "cd7bc946-cbb8-4966-b8c5-6edb16cc366e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:26:46.953954 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:46.953926 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:47.039666 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:47.039637 2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-ca-trust-extracted\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.039666 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:47.039662 2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-installation-pull-secrets\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.039666 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:47.039674 2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-registry-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.039878 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:47.039683 2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-bound-sa-token\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.039878 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:47.039692 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7cr7\" (UniqueName: \"kubernetes.io/projected/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-kube-api-access-j7cr7\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.039878 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:47.039703 2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd7bc946-cbb8-4966-b8c5-6edb16cc366e-image-registry-private-configuration\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:26:47.228456 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:47.228424 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74b85d8d69-gq2gp"] Jul 06 19:26:47.234620 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:47.234587 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74b85d8d69-gq2gp"] Jul 06 19:26:47.267751 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:47.267685 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:47.337723 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:47.337689 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:48.174831 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:48.174796 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd7bc946-cbb8-4966-b8c5-6edb16cc366e" path="/var/lib/kubelet/pods/cd7bc946-cbb8-4966-b8c5-6edb16cc366e/volumes" Jul 06 19:26:51.915846 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:51.915808 2572 generic.go:356] "Generic (PLEG): container finished" podID="2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112" containerID="0565ece6125e14cd409104123ce8993222aadde9ccd8fbebec71738866c724c2" exitCode=0 Jul 06 19:26:51.916358 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:51.915887 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" event={"ID":"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112","Type":"ContainerDied","Data":"0565ece6125e14cd409104123ce8993222aadde9ccd8fbebec71738866c724c2"} Jul 06 19:26:51.916358 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:51.916289 2572 scope.go:117] "RemoveContainer" containerID="0565ece6125e14cd409104123ce8993222aadde9ccd8fbebec71738866c724c2" Jul 06 19:26:52.921754 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:52.921718 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6qvnx" event={"ID":"2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112","Type":"ContainerStarted","Data":"3a88f26d7e8eaeb6027ee07a17e5576454e6af80f0663d306180c039f40ace09"} Jul 06 19:26:56.553560 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:56.553521 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:56.634626 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:26:56.634586 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:08.133806 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:27:08.133765 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:11.525413 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:27:11.525372 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:11.580698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:27:11.580658 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:12.000855 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:27:12.000819 2572 generic.go:356] "Generic (PLEG): container finished" podID="940db99f-f328-4608-b40e-31243191d438" containerID="962b0773b02d5f52f84e0173229d91c73864a2ce8ad1801c7bda40947b5cd65e" exitCode=0 Jul 06 19:27:12.001063 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:27:12.000896 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" event={"ID":"940db99f-f328-4608-b40e-31243191d438","Type":"ContainerDied","Data":"962b0773b02d5f52f84e0173229d91c73864a2ce8ad1801c7bda40947b5cd65e"} Jul 06 19:27:12.001286 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:27:12.001269 2572 scope.go:117] "RemoveContainer" containerID="962b0773b02d5f52f84e0173229d91c73864a2ce8ad1801c7bda40947b5cd65e" Jul 06 19:27:13.006702 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:27:13.006665 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-pkf7s" event={"ID":"940db99f-f328-4608-b40e-31243191d438","Type":"ContainerStarted","Data":"54412336d8453c193870d2833cd934b34aa3ced4ee94c0b3c905550debe53020"} Jul 06 19:27:15.505230 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:27:15.505190 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:15.560828 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:27:15.560791 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:13.193643 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:28:13.193550 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:13.284120 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:28:13.284078 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:12.639832 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:29:12.639751 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:17.145408 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:29:17.145368 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:49.903375 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:29:49.903344 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:29:49.907571 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:29:49.907551 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:29:49.912519 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:29:49.912494 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:29:49.914977 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:29:49.914955 2572 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 19:29:49.917263 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:29:49.917244 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:29:56.075831 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:29:56.075782 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:11.768617 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:30:11.768562 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:20.877197 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:20.877156 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:21.293584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:21.293524 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:23.993224 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:23.993185 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:24.051237 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:24.051197 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:24.162475 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:24.162438 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:24.448295 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:24.448200 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:27.291122 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:27.291083 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:30.272279 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:30.272245 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:30.380931 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:30.380867 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:30.439004 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:30.438967 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:30.962873 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:30.962833 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:31.027133 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:31.027099 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:41.606273 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:41.606242 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:41.706250 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:41.706219 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:41.768602 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:31:41.768563 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:37.181835 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:32:37.181801 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:37.254303 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:32:37.254265 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:49.208042 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:32:49.208000 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:49.305214 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:32:49.305178 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:52.921110 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:32:52.921075 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:59.064758 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:32:59.064724 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:59.506234 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:32:59.506191 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:59.604691 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:32:59.604626 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:05.816485 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:33:05.816437 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:17.034126 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:33:17.034086 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:17.143125 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:33:17.143082 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:34.651577 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:33:34.651535 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:34.756604 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:33:34.756566 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:57.591200 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:33:57.591090 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:57.653628 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:33:57.653585 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:01.661681 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:01.661646 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:02.503327 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:02.503289 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:05.282490 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:05.282450 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:05.384983 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:05.384934 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:06.562385 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:06.562348 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:06.663892 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:06.663850 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:07.273316 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:07.273284 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:07.361988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:07.361950 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:09.502178 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:09.502145 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:09.689218 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:09.689172 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:10.442706 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:10.442672 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:11.871972 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:11.871938 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:11.972663 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:11.972626 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:17.777388 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:17.777357 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:20.474079 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:20.474042 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:20.576844 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:20.576804 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:21.416924 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:21.416871 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:21.519458 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:21.519420 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:25.793281 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:25.793241 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:25.894429 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:25.894396 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:27.877751 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:27.877705 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:35.693284 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:35.693240 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:35.754021 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:35.753986 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:43.870362 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:43.870323 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:43.970706 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:43.970665 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:44.481357 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.481326 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:44.818373 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.818254 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:44.898957 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.898926 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:44.926098 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.926070 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b"] Jul 06 19:34:44.926375 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.926362 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd7bc946-cbb8-4966-b8c5-6edb16cc366e" containerName="registry" Jul 06 19:34:44.926417 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.926376 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd7bc946-cbb8-4966-b8c5-6edb16cc366e" containerName="registry" Jul 06 19:34:44.926450 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.926431 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd7bc946-cbb8-4966-b8c5-6edb16cc366e" containerName="registry" Jul 06 19:34:44.929377 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.929361 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:44.931799 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.931771 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 19:34:44.931993 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.931848 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 19:34:44.931993 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.931860 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 19:34:44.932668 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.932648 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pk84c\"" Jul 06 19:34:44.932806 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.932789 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 19:34:44.941591 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.941566 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b"] Jul 06 19:34:44.970162 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.970128 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf46798-5ef5-4e66-b54a-5928af921668-webhook-cert\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:44.970350 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.970180 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bf46798-5ef5-4e66-b54a-5928af921668-tmp\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:44.970350 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.970207 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8bf46798-5ef5-4e66-b54a-5928af921668-apiservice-cert\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:44.970350 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.970244 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8bf46798-5ef5-4e66-b54a-5928af921668-manifests\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:44.970350 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:44.970292 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mz9k\" (UniqueName: \"kubernetes.io/projected/8bf46798-5ef5-4e66-b54a-5928af921668-kube-api-access-6mz9k\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.046664 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.046624 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:45.071729 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.071634 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mz9k\" (UniqueName: \"kubernetes.io/projected/8bf46798-5ef5-4e66-b54a-5928af921668-kube-api-access-6mz9k\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.071729 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.071702 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf46798-5ef5-4e66-b54a-5928af921668-webhook-cert\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.071729 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.071729 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bf46798-5ef5-4e66-b54a-5928af921668-tmp\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.072015 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.071753 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8bf46798-5ef5-4e66-b54a-5928af921668-apiservice-cert\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.072015 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.071771 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8bf46798-5ef5-4e66-b54a-5928af921668-manifests\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.072241 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.072223 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8bf46798-5ef5-4e66-b54a-5928af921668-manifests\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.074167 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.074146 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bf46798-5ef5-4e66-b54a-5928af921668-tmp\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.074280 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.074262 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8bf46798-5ef5-4e66-b54a-5928af921668-apiservice-cert\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.074381 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.074364 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf46798-5ef5-4e66-b54a-5928af921668-webhook-cert\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.082277 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.082245 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mz9k\" (UniqueName: \"kubernetes.io/projected/8bf46798-5ef5-4e66-b54a-5928af921668-kube-api-access-6mz9k\") pod \"opendatahub-operator-controller-manager-849ccdb5-xh49b\" (UID: \"8bf46798-5ef5-4e66-b54a-5928af921668\") " pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.145954 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.145888 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:45.240699 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.240664 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:45.245587 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.245552 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:45.346287 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.346253 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:45.385395 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.385358 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b"] Jul 06 19:34:45.390203 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:34:45.390175 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf46798_5ef5_4e66_b54a_5928af921668.slice/crio-f282bd0e091537408a80242ffa31c0ca3089036d440cd1f0ea348880ce25fc17 WatchSource:0}: Error finding container f282bd0e091537408a80242ffa31c0ca3089036d440cd1f0ea348880ce25fc17: Status 404 returned error can't find the container with id f282bd0e091537408a80242ffa31c0ca3089036d440cd1f0ea348880ce25fc17 Jul 06 19:34:45.391509 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.391487 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:34:45.446259 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.446221 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:45.546655 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.546606 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:45.738021 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:45.737991 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" event={"ID":"8bf46798-5ef5-4e66-b54a-5928af921668","Type":"ContainerStarted","Data":"f282bd0e091537408a80242ffa31c0ca3089036d440cd1f0ea348880ce25fc17"} Jul 06 19:34:46.752654 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:46.752614 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:48.755473 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:48.755435 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" event={"ID":"8bf46798-5ef5-4e66-b54a-5928af921668","Type":"ContainerStarted","Data":"19daa032c71c87cc2c0dba75f1621d13d347161857081ba468e32594ac5ec6bc"} Jul 06 19:34:49.761305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:49.761264 2572 generic.go:356] "Generic (PLEG): container finished" podID="8bf46798-5ef5-4e66-b54a-5928af921668" containerID="19daa032c71c87cc2c0dba75f1621d13d347161857081ba468e32594ac5ec6bc" exitCode=0 Jul 06 19:34:49.761775 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:49.761347 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" event={"ID":"8bf46798-5ef5-4e66-b54a-5928af921668","Type":"ContainerDied","Data":"19daa032c71c87cc2c0dba75f1621d13d347161857081ba468e32594ac5ec6bc"} Jul 06 19:34:49.933862 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:49.933779 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:34:49.937594 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:49.937565 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:34:49.938034 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:49.938017 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:34:49.941840 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:49.941823 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:34:50.099401 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:50.099367 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:50.184497 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:50.184410 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:50.777261 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:50.777220 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" event={"ID":"8bf46798-5ef5-4e66-b54a-5928af921668","Type":"ContainerStarted","Data":"4fd91659dccbc1345f987f9346058b6888007f157b3b886590c0b9f2530c6446"} Jul 06 19:34:50.777677 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:50.777327 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:34:50.802638 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:34:50.802577 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" podStartSLOduration=2.434111748 podStartE2EDuration="6.802562354s" totalImagesPullingTime="3.275802723s" totalInitContainerRuntime="1.092647883s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:50.801521893 +0000 UTC m=+601.306806933" watchObservedRunningTime="2026-07-06 19:34:50.802562354 +0000 UTC m=+601.307847392" Jul 06 19:35:01.784922 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:01.784864 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 19:35:03.937351 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:03.937317 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:05.399831 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:05.399800 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:05.708116 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:05.708020 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:06.115422 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:06.115383 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:06.188754 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:06.188716 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:06.484293 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:06.484256 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:07.170934 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:07.170881 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:10.718844 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:10.718803 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:10.831019 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:10.830981 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:11.555516 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:11.555481 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:11.841435 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:11.841336 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:11.944189 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:11.944144 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:14.092704 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.092670 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68"] Jul 06 19:35:14.095657 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.095639 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.098580 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.098552 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xlll2\"" Jul 06 19:35:14.099479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.099461 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 19:35:14.099715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.099694 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:35:14.099785 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.099755 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 19:35:14.099843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.099755 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 19:35:14.099843 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.099823 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 19:35:14.106050 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.106025 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68"] Jul 06 19:35:14.226533 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.226495 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ee6796b4-e974-47e5-bf50-b5027d6a30c2-manager-config\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.226711 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.226541 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjvzg\" (UniqueName: \"kubernetes.io/projected/ee6796b4-e974-47e5-bf50-b5027d6a30c2-kube-api-access-gjvzg\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.226711 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.226611 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee6796b4-e974-47e5-bf50-b5027d6a30c2-cert\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.226711 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.226639 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee6796b4-e974-47e5-bf50-b5027d6a30c2-metrics-cert\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.327901 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.327862 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee6796b4-e974-47e5-bf50-b5027d6a30c2-cert\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.327901 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.327901 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee6796b4-e974-47e5-bf50-b5027d6a30c2-metrics-cert\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.328187 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.328018 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ee6796b4-e974-47e5-bf50-b5027d6a30c2-manager-config\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.328260 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.328188 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjvzg\" (UniqueName: \"kubernetes.io/projected/ee6796b4-e974-47e5-bf50-b5027d6a30c2-kube-api-access-gjvzg\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.328744 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.328719 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ee6796b4-e974-47e5-bf50-b5027d6a30c2-manager-config\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.330494 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.330472 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee6796b4-e974-47e5-bf50-b5027d6a30c2-cert\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.330680 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.330657 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee6796b4-e974-47e5-bf50-b5027d6a30c2-metrics-cert\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.350049 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.349974 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjvzg\" (UniqueName: \"kubernetes.io/projected/ee6796b4-e974-47e5-bf50-b5027d6a30c2-kube-api-access-gjvzg\") pod \"lws-controller-manager-67bb7fcf7f-cxv68\" (UID: \"ee6796b4-e974-47e5-bf50-b5027d6a30c2\") " pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.405883 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.405839 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:14.568229 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.568167 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68"] Jul 06 19:35:14.571922 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:35:14.571881 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee6796b4_e974_47e5_bf50_b5027d6a30c2.slice/crio-f32215f94fbfa9752e82e758dcf882ea328d009d227811e527c5ac713db3669e WatchSource:0}: Error finding container f32215f94fbfa9752e82e758dcf882ea328d009d227811e527c5ac713db3669e: Status 404 returned error can't find the container with id f32215f94fbfa9752e82e758dcf882ea328d009d227811e527c5ac713db3669e Jul 06 19:35:14.879791 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:14.879759 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" event={"ID":"ee6796b4-e974-47e5-bf50-b5027d6a30c2","Type":"ContainerStarted","Data":"f32215f94fbfa9752e82e758dcf882ea328d009d227811e527c5ac713db3669e"} Jul 06 19:35:15.396771 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.396733 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:15.430915 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.430869 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lwftm"] Jul 06 19:35:15.433720 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.433700 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 19:35:15.444478 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.444454 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 19:35:15.449069 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.449044 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-xz2w9\"" Jul 06 19:35:15.452481 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.452453 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lwftm"] Jul 06 19:35:15.461942 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.461890 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577"] Jul 06 19:35:15.464875 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.464832 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 19:35:15.471131 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.471104 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-q6t9p\"" Jul 06 19:35:15.471608 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.471587 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 19:35:15.494257 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.494221 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577"] Jul 06 19:35:15.502335 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.502269 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:15.540006 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.539966 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjc9j\" (UniqueName: \"kubernetes.io/projected/fad8e3b0-82a9-4d95-8807-aa3819bcb9e7-kube-api-access-mjc9j\") pod \"kserve-controller-manager-d6f4b65b8-lwftm\" (UID: \"fad8e3b0-82a9-4d95-8807-aa3819bcb9e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 19:35:15.540186 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.540027 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fad8e3b0-82a9-4d95-8807-aa3819bcb9e7-cert\") pod \"kserve-controller-manager-d6f4b65b8-lwftm\" (UID: \"fad8e3b0-82a9-4d95-8807-aa3819bcb9e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 19:35:15.601041 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.601006 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:15.641069 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.641029 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjc9j\" (UniqueName: \"kubernetes.io/projected/fad8e3b0-82a9-4d95-8807-aa3819bcb9e7-kube-api-access-mjc9j\") pod \"kserve-controller-manager-d6f4b65b8-lwftm\" (UID: \"fad8e3b0-82a9-4d95-8807-aa3819bcb9e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 19:35:15.641069 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.641070 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sksdm\" (UniqueName: \"kubernetes.io/projected/8f8ff396-377b-4cd6-a4f2-9b26caeab604-kube-api-access-sksdm\") pod \"llmisvc-controller-manager-777c6bd7d5-x2577\" (UID: \"8f8ff396-377b-4cd6-a4f2-9b26caeab604\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 19:35:15.641346 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.641102 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f8ff396-377b-4cd6-a4f2-9b26caeab604-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-x2577\" (UID: \"8f8ff396-377b-4cd6-a4f2-9b26caeab604\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 19:35:15.641346 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.641123 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fad8e3b0-82a9-4d95-8807-aa3819bcb9e7-cert\") pod \"kserve-controller-manager-d6f4b65b8-lwftm\" (UID: \"fad8e3b0-82a9-4d95-8807-aa3819bcb9e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 19:35:15.643703 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.643676 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fad8e3b0-82a9-4d95-8807-aa3819bcb9e7-cert\") pod \"kserve-controller-manager-d6f4b65b8-lwftm\" (UID: \"fad8e3b0-82a9-4d95-8807-aa3819bcb9e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 19:35:15.652146 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.652083 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjc9j\" (UniqueName: \"kubernetes.io/projected/fad8e3b0-82a9-4d95-8807-aa3819bcb9e7-kube-api-access-mjc9j\") pod \"kserve-controller-manager-d6f4b65b8-lwftm\" (UID: \"fad8e3b0-82a9-4d95-8807-aa3819bcb9e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 19:35:15.700058 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.700025 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:15.742218 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.742179 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sksdm\" (UniqueName: \"kubernetes.io/projected/8f8ff396-377b-4cd6-a4f2-9b26caeab604-kube-api-access-sksdm\") pod \"llmisvc-controller-manager-777c6bd7d5-x2577\" (UID: \"8f8ff396-377b-4cd6-a4f2-9b26caeab604\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 19:35:15.742408 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.742244 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f8ff396-377b-4cd6-a4f2-9b26caeab604-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-x2577\" (UID: \"8f8ff396-377b-4cd6-a4f2-9b26caeab604\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 19:35:15.744960 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.744936 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f8ff396-377b-4cd6-a4f2-9b26caeab604-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-x2577\" (UID: \"8f8ff396-377b-4cd6-a4f2-9b26caeab604\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 19:35:15.746840 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.746795 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 19:35:15.757979 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.757952 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sksdm\" (UniqueName: \"kubernetes.io/projected/8f8ff396-377b-4cd6-a4f2-9b26caeab604-kube-api-access-sksdm\") pod \"llmisvc-controller-manager-777c6bd7d5-x2577\" (UID: \"8f8ff396-377b-4cd6-a4f2-9b26caeab604\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 19:35:15.775536 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.775501 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 19:35:15.920448 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.920339 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lwftm"] Jul 06 19:35:15.924368 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:35:15.924331 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad8e3b0_82a9_4d95_8807_aa3819bcb9e7.slice/crio-1ce3b21904833337c0462e03e8144aaf0d29693901e2d74be8d66f1c9cadff6a WatchSource:0}: Error finding container 1ce3b21904833337c0462e03e8144aaf0d29693901e2d74be8d66f1c9cadff6a: Status 404 returned error can't find the container with id 1ce3b21904833337c0462e03e8144aaf0d29693901e2d74be8d66f1c9cadff6a Jul 06 19:35:15.943409 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:15.943360 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577"] Jul 06 19:35:15.946631 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:35:15.946569 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8f8ff396_377b_4cd6_a4f2_9b26caeab604.slice/crio-5244f709dd7f7bf854bfb54064906d7979465f7f138412075b1bdf6bcf32696e WatchSource:0}: Error finding container 5244f709dd7f7bf854bfb54064906d7979465f7f138412075b1bdf6bcf32696e: Status 404 returned error can't find the container with id 5244f709dd7f7bf854bfb54064906d7979465f7f138412075b1bdf6bcf32696e Jul 06 19:35:16.698037 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:16.697987 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:16.798945 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:16.798880 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:16.872929 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:16.872847 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:16.894491 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:16.894452 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" event={"ID":"8f8ff396-377b-4cd6-a4f2-9b26caeab604","Type":"ContainerStarted","Data":"5244f709dd7f7bf854bfb54064906d7979465f7f138412075b1bdf6bcf32696e"} Jul 06 19:35:16.896476 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:16.896409 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" event={"ID":"fad8e3b0-82a9-4d95-8807-aa3819bcb9e7","Type":"ContainerStarted","Data":"1ce3b21904833337c0462e03e8144aaf0d29693901e2d74be8d66f1c9cadff6a"} Jul 06 19:35:19.914111 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:19.914072 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" event={"ID":"8f8ff396-377b-4cd6-a4f2-9b26caeab604","Type":"ContainerStarted","Data":"250d54c59310e84a65fe3b58a9417f8384117606c47f71374131a5192f71fa44"} Jul 06 19:35:19.914558 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:19.914194 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 19:35:19.916057 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:19.916030 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" event={"ID":"ee6796b4-e974-47e5-bf50-b5027d6a30c2","Type":"ContainerStarted","Data":"d8d3eb447cbc397773ffc5049e479b6429103af14b292f37f5bfa4af3d440e31"} Jul 06 19:35:19.916203 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:19.916095 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:19.917813 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:19.917786 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" event={"ID":"fad8e3b0-82a9-4d95-8807-aa3819bcb9e7","Type":"ContainerStarted","Data":"36390896e13fd7bda86fad6dbaf04b34f3b55b5f193221cb0da329ebf1fd17a4"} Jul 06 19:35:19.917945 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:19.917932 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 19:35:19.933593 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:19.933534 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" podStartSLOduration=1.284432469 podStartE2EDuration="4.933517703s" totalImagesPullingTime="3.649085234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:19.932718418 +0000 UTC m=+630.438003456" watchObservedRunningTime="2026-07-06 19:35:19.933517703 +0000 UTC m=+630.438802773" Jul 06 19:35:19.952250 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:19.952192 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" podStartSLOduration=0.933829372 podStartE2EDuration="5.952175058s" totalImagesPullingTime="5.018345686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:19.950429687 +0000 UTC m=+630.455714741" watchObservedRunningTime="2026-07-06 19:35:19.952175058 +0000 UTC m=+630.457460096" Jul 06 19:35:19.969475 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:19.969415 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" podStartSLOduration=1.302055951 podStartE2EDuration="4.969395225s" totalImagesPullingTime="3.667339274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:19.968399676 +0000 UTC m=+630.473684717" watchObservedRunningTime="2026-07-06 19:35:19.969395225 +0000 UTC m=+630.474680262" Jul 06 19:35:30.924977 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:30.924946 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-67bb7fcf7f-cxv68" Jul 06 19:35:39.562407 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:39.562341 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:39.626323 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:39.626284 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:50.924978 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:50.924946 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 19:35:50.928001 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:35:50.927976 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 19:36:04.251161 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:04.251125 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:04.313210 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:04.313177 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:05.324022 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:05.323979 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:05.433758 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:05.433717 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:05.560091 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:05.560039 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:28.799855 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:28.799814 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:31.800480 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:31.800434 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:32.617869 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:32.617834 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:36.223778 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:36.223739 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:36.323980 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:36.323943 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:36.419378 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:36.419341 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:38.021151 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.021107 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:38.476256 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.476219 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m"] Jul 06 19:36:38.479419 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.479399 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" Jul 06 19:36:38.490698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.490676 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:36:38.490698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.490689 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 19:36:38.490698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.490676 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-x6x6b\"" Jul 06 19:36:38.491552 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.491537 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:36:38.509423 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.509390 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m"] Jul 06 19:36:38.552970 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.552935 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5dlx\" (UniqueName: \"kubernetes.io/projected/233283df-b179-4063-97ec-96608510f7d7-kube-api-access-t5dlx\") pod \"dns-operator-controller-manager-864dd78c4c-8jm6m\" (UID: \"233283df-b179-4063-97ec-96608510f7d7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" Jul 06 19:36:38.653869 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.653826 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5dlx\" (UniqueName: \"kubernetes.io/projected/233283df-b179-4063-97ec-96608510f7d7-kube-api-access-t5dlx\") pod \"dns-operator-controller-manager-864dd78c4c-8jm6m\" (UID: \"233283df-b179-4063-97ec-96608510f7d7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" Jul 06 19:36:38.663368 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.663337 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5dlx\" (UniqueName: \"kubernetes.io/projected/233283df-b179-4063-97ec-96608510f7d7-kube-api-access-t5dlx\") pod \"dns-operator-controller-manager-864dd78c4c-8jm6m\" (UID: \"233283df-b179-4063-97ec-96608510f7d7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" Jul 06 19:36:38.789322 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.789228 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" Jul 06 19:36:38.953105 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:38.953079 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m"] Jul 06 19:36:38.955888 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:36:38.955855 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod233283df_b179_4063_97ec_96608510f7d7.slice/crio-ac6008311ab18f98d01c57a313d297f6627513e5ddd043e41d382542fbc831c7 WatchSource:0}: Error finding container ac6008311ab18f98d01c57a313d297f6627513e5ddd043e41d382542fbc831c7: Status 404 returned error can't find the container with id ac6008311ab18f98d01c57a313d297f6627513e5ddd043e41d382542fbc831c7 Jul 06 19:36:39.193346 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:39.193266 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:39.268036 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:39.267990 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" event={"ID":"233283df-b179-4063-97ec-96608510f7d7","Type":"ContainerStarted","Data":"ac6008311ab18f98d01c57a313d297f6627513e5ddd043e41d382542fbc831c7"} Jul 06 19:36:39.467332 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:39.467296 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:41.266237 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:41.266190 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:41.279422 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:41.279387 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" event={"ID":"233283df-b179-4063-97ec-96608510f7d7","Type":"ContainerStarted","Data":"8597863ef737f47585763a2abeeb6d167c4cb6c9805d3ca60a4e9a0281ebac82"} Jul 06 19:36:41.279613 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:41.279463 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" Jul 06 19:36:41.324679 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:41.324612 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" podStartSLOduration=1.756336149 podStartE2EDuration="3.324595013s" totalImagesPullingTime="1.568258864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:36:41.322141127 +0000 UTC m=+711.827426165" watchObservedRunningTime="2026-07-06 19:36:41.324595013 +0000 UTC m=+711.829880049" Jul 06 19:36:43.225708 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.225630 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm"] Jul 06 19:36:43.227753 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.227738 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:36:43.231060 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.231041 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jt9tx\"" Jul 06 19:36:43.244258 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.244234 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm"] Jul 06 19:36:43.292452 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.292417 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjgbl\" (UniqueName: \"kubernetes.io/projected/6667ccac-fe61-4b51-b527-0d3d66f8f252-kube-api-access-xjgbl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzvnm\" (UID: \"6667ccac-fe61-4b51-b527-0d3d66f8f252\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:36:43.292616 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.292464 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6667ccac-fe61-4b51-b527-0d3d66f8f252-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzvnm\" (UID: \"6667ccac-fe61-4b51-b527-0d3d66f8f252\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:36:43.393555 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.393516 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjgbl\" (UniqueName: \"kubernetes.io/projected/6667ccac-fe61-4b51-b527-0d3d66f8f252-kube-api-access-xjgbl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzvnm\" (UID: \"6667ccac-fe61-4b51-b527-0d3d66f8f252\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:36:43.393758 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.393571 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6667ccac-fe61-4b51-b527-0d3d66f8f252-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzvnm\" (UID: \"6667ccac-fe61-4b51-b527-0d3d66f8f252\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:36:43.394058 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.394035 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6667ccac-fe61-4b51-b527-0d3d66f8f252-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzvnm\" (UID: \"6667ccac-fe61-4b51-b527-0d3d66f8f252\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:36:43.407555 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.407527 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjgbl\" (UniqueName: \"kubernetes.io/projected/6667ccac-fe61-4b51-b527-0d3d66f8f252-kube-api-access-xjgbl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzvnm\" (UID: \"6667ccac-fe61-4b51-b527-0d3d66f8f252\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:36:43.538264 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.538177 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:36:43.680378 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:43.680342 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm"] Jul 06 19:36:43.684090 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:36:43.684051 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6667ccac_fe61_4b51_b527_0d3d66f8f252.slice/crio-d5f46486bfeb00b8ff0e7cd03e468c38f44c96406f4fb5f26d88d5acac944d48 WatchSource:0}: Error finding container d5f46486bfeb00b8ff0e7cd03e468c38f44c96406f4fb5f26d88d5acac944d48: Status 404 returned error can't find the container with id d5f46486bfeb00b8ff0e7cd03e468c38f44c96406f4fb5f26d88d5acac944d48 Jul 06 19:36:44.295074 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:44.295042 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" event={"ID":"6667ccac-fe61-4b51-b527-0d3d66f8f252","Type":"ContainerStarted","Data":"d5f46486bfeb00b8ff0e7cd03e468c38f44c96406f4fb5f26d88d5acac944d48"} Jul 06 19:36:44.421338 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:44.421294 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:44.523784 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:44.523741 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:49.323218 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:49.323183 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" event={"ID":"6667ccac-fe61-4b51-b527-0d3d66f8f252","Type":"ContainerStarted","Data":"eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5"} Jul 06 19:36:49.323614 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:49.323303 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:36:49.351222 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:49.351164 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" podStartSLOduration=1.189990743 podStartE2EDuration="6.35114618s" totalImagesPullingTime="5.161155437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:36:49.349423428 +0000 UTC m=+719.854708472" watchObservedRunningTime="2026-07-06 19:36:49.35114618 +0000 UTC m=+719.856431218" Jul 06 19:36:49.804360 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:49.804321 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:49.874193 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:49.874163 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:50.206786 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:50.206684 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:50.303896 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:50.303863 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:51.631559 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:51.631521 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:51.849232 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:51.849192 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:52.286384 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:52.286356 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" Jul 06 19:36:53.254370 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:53.254316 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:53.358057 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:53.358028 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:57.343163 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:57.343129 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:57.445442 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:36:57.445406 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:00.330479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:00.330442 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:37:01.453800 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.453765 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf"] Jul 06 19:37:01.457336 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.457319 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:01.472292 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.472259 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf"] Jul 06 19:37:01.560491 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.560448 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcf43f52-5d0e-4136-a56a-77765aee7f39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hnsxf\" (UID: \"bcf43f52-5d0e-4136-a56a-77765aee7f39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:01.560709 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.560593 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnbvc\" (UniqueName: \"kubernetes.io/projected/bcf43f52-5d0e-4136-a56a-77765aee7f39-kube-api-access-wnbvc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hnsxf\" (UID: \"bcf43f52-5d0e-4136-a56a-77765aee7f39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:01.572811 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.572769 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf"] Jul 06 19:37:01.573086 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:01.573065 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-wnbvc], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" podUID="bcf43f52-5d0e-4136-a56a-77765aee7f39" Jul 06 19:37:01.599994 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.599949 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs"] Jul 06 19:37:01.603323 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.603296 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:01.617555 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.617529 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs"] Jul 06 19:37:01.661312 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.661270 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnbvc\" (UniqueName: \"kubernetes.io/projected/bcf43f52-5d0e-4136-a56a-77765aee7f39-kube-api-access-wnbvc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hnsxf\" (UID: \"bcf43f52-5d0e-4136-a56a-77765aee7f39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:01.661493 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.661334 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcf43f52-5d0e-4136-a56a-77765aee7f39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hnsxf\" (UID: \"bcf43f52-5d0e-4136-a56a-77765aee7f39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:01.661684 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.661661 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcf43f52-5d0e-4136-a56a-77765aee7f39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hnsxf\" (UID: \"bcf43f52-5d0e-4136-a56a-77765aee7f39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:01.670417 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.670392 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnbvc\" (UniqueName: \"kubernetes.io/projected/bcf43f52-5d0e-4136-a56a-77765aee7f39-kube-api-access-wnbvc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hnsxf\" (UID: \"bcf43f52-5d0e-4136-a56a-77765aee7f39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:01.762613 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.762573 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2wrq\" (UniqueName: \"kubernetes.io/projected/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-kube-api-access-g2wrq\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcsrs\" (UID: \"f9dd7ec2-1964-4d58-9c61-67ab2caafc9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:01.762791 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.762639 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcsrs\" (UID: \"f9dd7ec2-1964-4d58-9c61-67ab2caafc9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:01.864058 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.864022 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2wrq\" (UniqueName: \"kubernetes.io/projected/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-kube-api-access-g2wrq\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcsrs\" (UID: \"f9dd7ec2-1964-4d58-9c61-67ab2caafc9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:01.864231 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.864076 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcsrs\" (UID: \"f9dd7ec2-1964-4d58-9c61-67ab2caafc9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:01.864448 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.864430 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcsrs\" (UID: \"f9dd7ec2-1964-4d58-9c61-67ab2caafc9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:01.875093 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.875062 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2wrq\" (UniqueName: \"kubernetes.io/projected/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-kube-api-access-g2wrq\") pod \"kuadrant-operator-controller-manager-55467d9f7-jcsrs\" (UID: \"f9dd7ec2-1964-4d58-9c61-67ab2caafc9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:01.913834 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:01.913792 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:02.055341 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.053527 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs"] Jul 06 19:37:02.060186 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.060146 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs"] Jul 06 19:37:02.073113 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.073081 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf"] Jul 06 19:37:02.077139 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.077107 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld"] Jul 06 19:37:02.082628 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.082596 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf"] Jul 06 19:37:02.082788 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.082725 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:02.092240 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.092170 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm"] Jul 06 19:37:02.092528 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.092429 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" podUID="6667ccac-fe61-4b51-b527-0d3d66f8f252" containerName="manager" containerID="cri-o://eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5" gracePeriod=2 Jul 06 19:37:02.096506 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.095967 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld"] Jul 06 19:37:02.116631 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.116601 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm"] Jul 06 19:37:02.127330 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.127098 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc"] Jul 06 19:37:02.127638 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.127622 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6667ccac-fe61-4b51-b527-0d3d66f8f252" containerName="manager" Jul 06 19:37:02.127705 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.127641 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6667ccac-fe61-4b51-b527-0d3d66f8f252" containerName="manager" Jul 06 19:37:02.127757 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.127742 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6667ccac-fe61-4b51-b527-0d3d66f8f252" containerName="manager" Jul 06 19:37:02.131223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.131197 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:02.133615 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:02.133356 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xzvnm\" is forbidden: User \"system:node:ip-10-0-131-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-35.ec2.internal' and this object" podUID="6667ccac-fe61-4b51-b527-0d3d66f8f252" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:37:02.144671 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.144639 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc"] Jul 06 19:37:02.163171 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:02.163124 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xzvnm\" is forbidden: User \"system:node:ip-10-0-131-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-35.ec2.internal' and this object" podUID="6667ccac-fe61-4b51-b527-0d3d66f8f252" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:37:02.206311 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:02.206279 2572 log.go:32] "RunPodSandbox from runtime service failed" err=< Jul 06 19:37:02.206311 ip-10-0-131-35 kubenswrapper[2572]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-jcsrs_kuadrant-system_f9dd7ec2-1964-4d58-9c61-67ab2caafc9f_0(9e70e5fbb52599f1ec2a468bf414a4acd5d92cb37e8985280a8b4f4fe574410b): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-jcsrs to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"9e70e5fbb52599f1ec2a468bf414a4acd5d92cb37e8985280a8b4f4fe574410b" Netns:"/var/run/netns/9455de84-d413-4ff9-8beb-3864372630a1" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-jcsrs;K8S_POD_INFRA_CONTAINER_ID=9e70e5fbb52599f1ec2a468bf414a4acd5d92cb37e8985280a8b4f4fe574410b;K8S_POD_UID=f9dd7ec2-1964-4d58-9c61-67ab2caafc9f" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f]: error setting the networks status: SetPodNetworkStatusAnnotation: failed to update the pod kuadrant-operator-controller-manager-55467d9f7-jcsrs in out of cluster comm: SetNetworkStatus: failed to update the pod kuadrant-operator-controller-manager-55467d9f7-jcsrs in out of cluster comm: status update failed for pod /: pods "kuadrant-operator-controller-manager-55467d9f7-jcsrs" not found Jul 06 19:37:02.206311 ip-10-0-131-35 kubenswrapper[2572]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 06 19:37:02.206311 ip-10-0-131-35 kubenswrapper[2572]: > Jul 06 19:37:02.206466 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:02.206342 2572 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=< Jul 06 19:37:02.206466 ip-10-0-131-35 kubenswrapper[2572]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-jcsrs_kuadrant-system_f9dd7ec2-1964-4d58-9c61-67ab2caafc9f_0(9e70e5fbb52599f1ec2a468bf414a4acd5d92cb37e8985280a8b4f4fe574410b): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-jcsrs to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"9e70e5fbb52599f1ec2a468bf414a4acd5d92cb37e8985280a8b4f4fe574410b" Netns:"/var/run/netns/9455de84-d413-4ff9-8beb-3864372630a1" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-jcsrs;K8S_POD_INFRA_CONTAINER_ID=9e70e5fbb52599f1ec2a468bf414a4acd5d92cb37e8985280a8b4f4fe574410b;K8S_POD_UID=f9dd7ec2-1964-4d58-9c61-67ab2caafc9f" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f]: error setting the networks status: SetPodNetworkStatusAnnotation: failed to update the pod kuadrant-operator-controller-manager-55467d9f7-jcsrs in out of cluster comm: SetNetworkStatus: failed to update the pod kuadrant-operator-controller-manager-55467d9f7-jcsrs in out of cluster comm: status update failed for pod /: pods "kuadrant-operator-controller-manager-55467d9f7-jcsrs" not found Jul 06 19:37:02.206466 ip-10-0-131-35 kubenswrapper[2572]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 06 19:37:02.206466 ip-10-0-131-35 kubenswrapper[2572]: > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:02.267440 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.267402 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl7fb\" (UniqueName: \"kubernetes.io/projected/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-kube-api-access-vl7fb\") pod \"kuadrant-operator-controller-manager-55467d9f7-l6nld\" (UID: \"0f26e631-46f4-4b3e-8ee3-3a251e5c8305\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:02.267580 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.267465 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-l6nld\" (UID: \"0f26e631-46f4-4b3e-8ee3-3a251e5c8305\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:02.267580 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.267530 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a87c8d6e-210c-4472-b6d1-1ae0d841985c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4mwbc\" (UID: \"a87c8d6e-210c-4472-b6d1-1ae0d841985c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:02.267580 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.267573 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgpvg\" (UniqueName: \"kubernetes.io/projected/a87c8d6e-210c-4472-b6d1-1ae0d841985c-kube-api-access-cgpvg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4mwbc\" (UID: \"a87c8d6e-210c-4472-b6d1-1ae0d841985c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:02.331289 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.331260 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:37:02.368526 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.368493 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl7fb\" (UniqueName: \"kubernetes.io/projected/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-kube-api-access-vl7fb\") pod \"kuadrant-operator-controller-manager-55467d9f7-l6nld\" (UID: \"0f26e631-46f4-4b3e-8ee3-3a251e5c8305\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:02.368683 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.368534 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-l6nld\" (UID: \"0f26e631-46f4-4b3e-8ee3-3a251e5c8305\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:02.368683 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.368574 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a87c8d6e-210c-4472-b6d1-1ae0d841985c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4mwbc\" (UID: \"a87c8d6e-210c-4472-b6d1-1ae0d841985c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:02.368683 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.368608 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgpvg\" (UniqueName: \"kubernetes.io/projected/a87c8d6e-210c-4472-b6d1-1ae0d841985c-kube-api-access-cgpvg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4mwbc\" (UID: \"a87c8d6e-210c-4472-b6d1-1ae0d841985c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:02.368962 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.368894 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-l6nld\" (UID: \"0f26e631-46f4-4b3e-8ee3-3a251e5c8305\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:02.369074 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.368998 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a87c8d6e-210c-4472-b6d1-1ae0d841985c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4mwbc\" (UID: \"a87c8d6e-210c-4472-b6d1-1ae0d841985c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:02.380111 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.380074 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl7fb\" (UniqueName: \"kubernetes.io/projected/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-kube-api-access-vl7fb\") pod \"kuadrant-operator-controller-manager-55467d9f7-l6nld\" (UID: \"0f26e631-46f4-4b3e-8ee3-3a251e5c8305\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:02.380268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.380221 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgpvg\" (UniqueName: \"kubernetes.io/projected/a87c8d6e-210c-4472-b6d1-1ae0d841985c-kube-api-access-cgpvg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4mwbc\" (UID: \"a87c8d6e-210c-4472-b6d1-1ae0d841985c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:02.383952 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.383902 2572 generic.go:356] "Generic (PLEG): container finished" podID="6667ccac-fe61-4b51-b527-0d3d66f8f252" containerID="eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5" exitCode=0 Jul 06 19:37:02.384078 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.383963 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzvnm" Jul 06 19:37:02.384078 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.384011 2572 scope.go:117] "RemoveContainer" containerID="eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5" Jul 06 19:37:02.384195 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.384095 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:02.384195 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.384168 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:02.391047 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.391022 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:02.392303 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.392287 2572 scope.go:117] "RemoveContainer" containerID="eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5" Jul 06 19:37:02.392551 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:02.392531 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5\": container with ID starting with eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5 not found: ID does not exist" containerID="eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5" Jul 06 19:37:02.392613 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.392559 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5"} err="failed to get container status \"eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5\": rpc error: code = NotFound desc = could not find container \"eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5\": container with ID starting with eb465077baf5704e589261e0c7d5738a4ba461a515e63987e60f2b7d4c3037f5 not found: ID does not exist" Jul 06 19:37:02.393172 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:02.393151 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jcsrs\" is forbidden: User \"system:node:ip-10-0-131-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-35.ec2.internal' and this object" podUID="f9dd7ec2-1964-4d58-9c61-67ab2caafc9f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:02.395798 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.395783 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:02.397737 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:02.397714 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-hnsxf\" is forbidden: User \"system:node:ip-10-0-131-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-35.ec2.internal' and this object" podUID="bcf43f52-5d0e-4136-a56a-77765aee7f39" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:02.398845 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.398831 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:02.399416 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:02.399397 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jcsrs\" is forbidden: User \"system:node:ip-10-0-131-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-35.ec2.internal' and this object" podUID="f9dd7ec2-1964-4d58-9c61-67ab2caafc9f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:02.469875 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.469845 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjgbl\" (UniqueName: \"kubernetes.io/projected/6667ccac-fe61-4b51-b527-0d3d66f8f252-kube-api-access-xjgbl\") pod \"6667ccac-fe61-4b51-b527-0d3d66f8f252\" (UID: \"6667ccac-fe61-4b51-b527-0d3d66f8f252\") " Jul 06 19:37:02.469875 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.469880 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6667ccac-fe61-4b51-b527-0d3d66f8f252-extensions-socket-volume\") pod \"6667ccac-fe61-4b51-b527-0d3d66f8f252\" (UID: \"6667ccac-fe61-4b51-b527-0d3d66f8f252\") " Jul 06 19:37:02.470447 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.470413 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6667ccac-fe61-4b51-b527-0d3d66f8f252-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6667ccac-fe61-4b51-b527-0d3d66f8f252" (UID: "6667ccac-fe61-4b51-b527-0d3d66f8f252"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:02.472439 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.472416 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:02.472584 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.472480 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6667ccac-fe61-4b51-b527-0d3d66f8f252-kube-api-access-xjgbl" (OuterVolumeSpecName: "kube-api-access-xjgbl") pod "6667ccac-fe61-4b51-b527-0d3d66f8f252" (UID: "6667ccac-fe61-4b51-b527-0d3d66f8f252"). InnerVolumeSpecName "kube-api-access-xjgbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:37:02.550364 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.550338 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld"] Jul 06 19:37:02.553324 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:37:02.553293 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f26e631_46f4_4b3e_8ee3_3a251e5c8305.slice/crio-89e5b75570bc5546dfddd88265bc4a40a89e35049eb9c7e97872d9528e2b574e WatchSource:0}: Error finding container 89e5b75570bc5546dfddd88265bc4a40a89e35049eb9c7e97872d9528e2b574e: Status 404 returned error can't find the container with id 89e5b75570bc5546dfddd88265bc4a40a89e35049eb9c7e97872d9528e2b574e Jul 06 19:37:02.571137 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.571113 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnbvc\" (UniqueName: \"kubernetes.io/projected/bcf43f52-5d0e-4136-a56a-77765aee7f39-kube-api-access-wnbvc\") pod \"bcf43f52-5d0e-4136-a56a-77765aee7f39\" (UID: \"bcf43f52-5d0e-4136-a56a-77765aee7f39\") " Jul 06 19:37:02.571253 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.571142 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2wrq\" (UniqueName: \"kubernetes.io/projected/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-kube-api-access-g2wrq\") pod \"f9dd7ec2-1964-4d58-9c61-67ab2caafc9f\" (UID: \"f9dd7ec2-1964-4d58-9c61-67ab2caafc9f\") " Jul 06 19:37:02.571253 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.571181 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-extensions-socket-volume\") pod \"f9dd7ec2-1964-4d58-9c61-67ab2caafc9f\" (UID: \"f9dd7ec2-1964-4d58-9c61-67ab2caafc9f\") " Jul 06 19:37:02.571253 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.571225 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcf43f52-5d0e-4136-a56a-77765aee7f39-extensions-socket-volume\") pod \"bcf43f52-5d0e-4136-a56a-77765aee7f39\" (UID: \"bcf43f52-5d0e-4136-a56a-77765aee7f39\") " Jul 06 19:37:02.571667 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.571482 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjgbl\" (UniqueName: \"kubernetes.io/projected/6667ccac-fe61-4b51-b527-0d3d66f8f252-kube-api-access-xjgbl\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:02.571667 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.571506 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6667ccac-fe61-4b51-b527-0d3d66f8f252-extensions-socket-volume\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:02.571667 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.571586 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f9dd7ec2-1964-4d58-9c61-67ab2caafc9f" (UID: "f9dd7ec2-1964-4d58-9c61-67ab2caafc9f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:02.571883 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.571679 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf43f52-5d0e-4136-a56a-77765aee7f39-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bcf43f52-5d0e-4136-a56a-77765aee7f39" (UID: "bcf43f52-5d0e-4136-a56a-77765aee7f39"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:02.575119 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.575057 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf43f52-5d0e-4136-a56a-77765aee7f39-kube-api-access-wnbvc" (OuterVolumeSpecName: "kube-api-access-wnbvc") pod "bcf43f52-5d0e-4136-a56a-77765aee7f39" (UID: "bcf43f52-5d0e-4136-a56a-77765aee7f39"). InnerVolumeSpecName "kube-api-access-wnbvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:37:02.575119 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.575098 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-kube-api-access-g2wrq" (OuterVolumeSpecName: "kube-api-access-g2wrq") pod "f9dd7ec2-1964-4d58-9c61-67ab2caafc9f" (UID: "f9dd7ec2-1964-4d58-9c61-67ab2caafc9f"). InnerVolumeSpecName "kube-api-access-g2wrq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:37:02.622037 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.622011 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc"] Jul 06 19:37:02.625153 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:37:02.625126 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda87c8d6e_210c_4472_b6d1_1ae0d841985c.slice/crio-3bc30eb64a740c85ab5dfc6c500703088b2ad9f2a2c303392305d731d86a3a96 WatchSource:0}: Error finding container 3bc30eb64a740c85ab5dfc6c500703088b2ad9f2a2c303392305d731d86a3a96: Status 404 returned error can't find the container with id 3bc30eb64a740c85ab5dfc6c500703088b2ad9f2a2c303392305d731d86a3a96 Jul 06 19:37:02.672149 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.672121 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnbvc\" (UniqueName: \"kubernetes.io/projected/bcf43f52-5d0e-4136-a56a-77765aee7f39-kube-api-access-wnbvc\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:02.672149 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.672147 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2wrq\" (UniqueName: \"kubernetes.io/projected/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-kube-api-access-g2wrq\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:02.672319 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.672158 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f-extensions-socket-volume\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:02.672319 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:02.672169 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcf43f52-5d0e-4136-a56a-77765aee7f39-extensions-socket-volume\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:02.704951 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:02.704901 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-hnsxf\" is forbidden: User \"system:node:ip-10-0-131-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-35.ec2.internal' and this object" podUID="bcf43f52-5d0e-4136-a56a-77765aee7f39" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:02.706766 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:02.706744 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jcsrs\" is forbidden: User \"system:node:ip-10-0-131-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-35.ec2.internal' and this object" podUID="f9dd7ec2-1964-4d58-9c61-67ab2caafc9f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:03.391983 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:03.391936 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" event={"ID":"a87c8d6e-210c-4472-b6d1-1ae0d841985c","Type":"ContainerStarted","Data":"2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3"} Jul 06 19:37:03.391983 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:03.391990 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" event={"ID":"a87c8d6e-210c-4472-b6d1-1ae0d841985c","Type":"ContainerStarted","Data":"3bc30eb64a740c85ab5dfc6c500703088b2ad9f2a2c303392305d731d86a3a96"} Jul 06 19:37:03.392243 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:03.392071 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:03.393529 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:03.393496 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" event={"ID":"0f26e631-46f4-4b3e-8ee3-3a251e5c8305","Type":"ContainerStarted","Data":"51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09"} Jul 06 19:37:03.393657 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:03.393550 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:03.393657 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:03.393646 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:03.393782 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:03.393677 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" event={"ID":"0f26e631-46f4-4b3e-8ee3-3a251e5c8305","Type":"ContainerStarted","Data":"89e5b75570bc5546dfddd88265bc4a40a89e35049eb9c7e97872d9528e2b574e"} Jul 06 19:37:03.393782 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:03.393706 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:03.394145 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:03.394122 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-hnsxf\" is forbidden: User \"system:node:ip-10-0-131-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-35.ec2.internal' and this object" podUID="bcf43f52-5d0e-4136-a56a-77765aee7f39" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:03.415726 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:03.415687 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jcsrs\" is forbidden: User \"system:node:ip-10-0-131-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-35.ec2.internal' and this object" podUID="f9dd7ec2-1964-4d58-9c61-67ab2caafc9f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:03.415917 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:03.415865 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" podStartSLOduration=1.415849737 podStartE2EDuration="1.415849737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:37:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:37:03.413537471 +0000 UTC m=+733.918822510" watchObservedRunningTime="2026-07-06 19:37:03.415849737 +0000 UTC m=+733.921134775" Jul 06 19:37:03.417403 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:03.417381 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jcsrs\" is forbidden: User \"system:node:ip-10-0-131-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-35.ec2.internal' and this object" podUID="f9dd7ec2-1964-4d58-9c61-67ab2caafc9f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jcsrs" Jul 06 19:37:03.418955 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:03.418938 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-hnsxf\" is forbidden: User \"system:node:ip-10-0-131-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-35.ec2.internal' and this object" podUID="bcf43f52-5d0e-4136-a56a-77765aee7f39" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hnsxf" Jul 06 19:37:03.443831 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:03.443775 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" podStartSLOduration=1.44375431 podStartE2EDuration="1.44375431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:37:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:37:03.441979806 +0000 UTC m=+733.947264845" watchObservedRunningTime="2026-07-06 19:37:03.44375431 +0000 UTC m=+733.949039348" Jul 06 19:37:04.175845 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:04.175806 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6667ccac-fe61-4b51-b527-0d3d66f8f252" path="/var/lib/kubelet/pods/6667ccac-fe61-4b51-b527-0d3d66f8f252/volumes" Jul 06 19:37:04.176274 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:04.176184 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcf43f52-5d0e-4136-a56a-77765aee7f39" path="/var/lib/kubelet/pods/bcf43f52-5d0e-4136-a56a-77765aee7f39/volumes" Jul 06 19:37:04.176424 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:04.176410 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9dd7ec2-1964-4d58-9c61-67ab2caafc9f" path="/var/lib/kubelet/pods/f9dd7ec2-1964-4d58-9c61-67ab2caafc9f/volumes" Jul 06 19:37:04.260282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:04.260247 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:04.343590 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:04.343558 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:05.742062 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:05.742019 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:05.808150 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:05.808112 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:07.937171 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:07.937137 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:08.013041 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:08.013000 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:08.177859 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:08.177828 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:08.273286 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:08.273251 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:11.740263 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:11.740231 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:11.807100 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:11.807058 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:14.400272 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.400241 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:14.400655 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.400435 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:14.528083 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.528046 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc"] Jul 06 19:37:14.528296 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.528273 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" podUID="a87c8d6e-210c-4472-b6d1-1ae0d841985c" containerName="manager" containerID="cri-o://2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3" gracePeriod=10 Jul 06 19:37:14.793087 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.793057 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:14.870763 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.870727 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh"] Jul 06 19:37:14.871112 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.871089 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a87c8d6e-210c-4472-b6d1-1ae0d841985c" containerName="manager" Jul 06 19:37:14.871112 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.871105 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a87c8d6e-210c-4472-b6d1-1ae0d841985c" containerName="manager" Jul 06 19:37:14.871288 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.871199 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a87c8d6e-210c-4472-b6d1-1ae0d841985c" containerName="manager" Jul 06 19:37:14.874607 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.874580 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgpvg\" (UniqueName: \"kubernetes.io/projected/a87c8d6e-210c-4472-b6d1-1ae0d841985c-kube-api-access-cgpvg\") pod \"a87c8d6e-210c-4472-b6d1-1ae0d841985c\" (UID: \"a87c8d6e-210c-4472-b6d1-1ae0d841985c\") " Jul 06 19:37:14.874766 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.874643 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a87c8d6e-210c-4472-b6d1-1ae0d841985c-extensions-socket-volume\") pod \"a87c8d6e-210c-4472-b6d1-1ae0d841985c\" (UID: \"a87c8d6e-210c-4472-b6d1-1ae0d841985c\") " Jul 06 19:37:14.875081 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.875059 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a87c8d6e-210c-4472-b6d1-1ae0d841985c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a87c8d6e-210c-4472-b6d1-1ae0d841985c" (UID: "a87c8d6e-210c-4472-b6d1-1ae0d841985c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:14.876822 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.876796 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a87c8d6e-210c-4472-b6d1-1ae0d841985c-kube-api-access-cgpvg" (OuterVolumeSpecName: "kube-api-access-cgpvg") pod "a87c8d6e-210c-4472-b6d1-1ae0d841985c" (UID: "a87c8d6e-210c-4472-b6d1-1ae0d841985c"). InnerVolumeSpecName "kube-api-access-cgpvg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:37:14.877732 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.877702 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:37:14.936330 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.936292 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh"] Jul 06 19:37:14.975426 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.975392 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c0f207d-d493-452b-a771-c127bebe6843-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7hxhh\" (UID: \"3c0f207d-d493-452b-a771-c127bebe6843\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:37:14.975596 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.975453 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-776h9\" (UniqueName: \"kubernetes.io/projected/3c0f207d-d493-452b-a771-c127bebe6843-kube-api-access-776h9\") pod \"kuadrant-operator-controller-manager-86878c769f-7hxhh\" (UID: \"3c0f207d-d493-452b-a771-c127bebe6843\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:37:14.975596 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.975523 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgpvg\" (UniqueName: \"kubernetes.io/projected/a87c8d6e-210c-4472-b6d1-1ae0d841985c-kube-api-access-cgpvg\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:14.975596 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:14.975542 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a87c8d6e-210c-4472-b6d1-1ae0d841985c-extensions-socket-volume\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:15.076396 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.076355 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-776h9\" (UniqueName: \"kubernetes.io/projected/3c0f207d-d493-452b-a771-c127bebe6843-kube-api-access-776h9\") pod \"kuadrant-operator-controller-manager-86878c769f-7hxhh\" (UID: \"3c0f207d-d493-452b-a771-c127bebe6843\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:37:15.076569 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.076459 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c0f207d-d493-452b-a771-c127bebe6843-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7hxhh\" (UID: \"3c0f207d-d493-452b-a771-c127bebe6843\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:37:15.076869 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.076849 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c0f207d-d493-452b-a771-c127bebe6843-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7hxhh\" (UID: \"3c0f207d-d493-452b-a771-c127bebe6843\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:37:15.087625 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.087595 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-776h9\" (UniqueName: \"kubernetes.io/projected/3c0f207d-d493-452b-a771-c127bebe6843-kube-api-access-776h9\") pod \"kuadrant-operator-controller-manager-86878c769f-7hxhh\" (UID: \"3c0f207d-d493-452b-a771-c127bebe6843\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:37:15.195021 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.194992 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:37:15.342816 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.342767 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh"] Jul 06 19:37:15.345130 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:37:15.345103 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0f207d_d493_452b_a771_c127bebe6843.slice/crio-5df48ac0c4524a6045476c881eb619465f7f6b57d5f4413402a7d552cf1ddcd2 WatchSource:0}: Error finding container 5df48ac0c4524a6045476c881eb619465f7f6b57d5f4413402a7d552cf1ddcd2: Status 404 returned error can't find the container with id 5df48ac0c4524a6045476c881eb619465f7f6b57d5f4413402a7d552cf1ddcd2 Jul 06 19:37:15.449877 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.449841 2572 generic.go:356] "Generic (PLEG): container finished" podID="a87c8d6e-210c-4472-b6d1-1ae0d841985c" containerID="2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3" exitCode=0 Jul 06 19:37:15.450352 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.449889 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" Jul 06 19:37:15.450352 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.449940 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" event={"ID":"a87c8d6e-210c-4472-b6d1-1ae0d841985c","Type":"ContainerDied","Data":"2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3"} Jul 06 19:37:15.450352 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.450007 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc" event={"ID":"a87c8d6e-210c-4472-b6d1-1ae0d841985c","Type":"ContainerDied","Data":"3bc30eb64a740c85ab5dfc6c500703088b2ad9f2a2c303392305d731d86a3a96"} Jul 06 19:37:15.450352 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.450043 2572 scope.go:117] "RemoveContainer" containerID="2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3" Jul 06 19:37:15.451978 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.451899 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" event={"ID":"3c0f207d-d493-452b-a771-c127bebe6843","Type":"ContainerStarted","Data":"260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8"} Jul 06 19:37:15.452117 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.452013 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:37:15.452117 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.452034 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" event={"ID":"3c0f207d-d493-452b-a771-c127bebe6843","Type":"ContainerStarted","Data":"5df48ac0c4524a6045476c881eb619465f7f6b57d5f4413402a7d552cf1ddcd2"} Jul 06 19:37:15.459675 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.459654 2572 scope.go:117] "RemoveContainer" containerID="2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3" Jul 06 19:37:15.459997 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:15.459966 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3\": container with ID starting with 2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3 not found: ID does not exist" containerID="2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3" Jul 06 19:37:15.460094 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.460008 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3"} err="failed to get container status \"2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3\": rpc error: code = NotFound desc = could not find container \"2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3\": container with ID starting with 2801a767fd75c1dc94e9fc8a2fc6d4c5ce8ad8cab16697b9dbebc8ae2181d0a3 not found: ID does not exist" Jul 06 19:37:15.480226 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.480169 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" podStartSLOduration=1.480151952 podStartE2EDuration="1.480151952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:37:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:37:15.478251609 +0000 UTC m=+745.983536648" watchObservedRunningTime="2026-07-06 19:37:15.480151952 +0000 UTC m=+745.985436990" Jul 06 19:37:15.508045 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.507940 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc"] Jul 06 19:37:15.509880 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:15.509846 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4mwbc"] Jul 06 19:37:16.174852 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:16.174816 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a87c8d6e-210c-4472-b6d1-1ae0d841985c" path="/var/lib/kubelet/pods/a87c8d6e-210c-4472-b6d1-1ae0d841985c/volumes" Jul 06 19:37:16.344500 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:16.344462 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:16.440973 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:16.440873 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:26.460199 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:26.460168 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:37:26.515923 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:26.515883 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld"] Jul 06 19:37:26.516143 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:26.516117 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" podUID="0f26e631-46f4-4b3e-8ee3-3a251e5c8305" containerName="manager" containerID="cri-o://51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09" gracePeriod=10 Jul 06 19:37:26.772418 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:26.772388 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:26.881459 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:26.881409 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vl7fb\" (UniqueName: \"kubernetes.io/projected/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-kube-api-access-vl7fb\") pod \"0f26e631-46f4-4b3e-8ee3-3a251e5c8305\" (UID: \"0f26e631-46f4-4b3e-8ee3-3a251e5c8305\") " Jul 06 19:37:26.881633 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:26.881516 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-extensions-socket-volume\") pod \"0f26e631-46f4-4b3e-8ee3-3a251e5c8305\" (UID: \"0f26e631-46f4-4b3e-8ee3-3a251e5c8305\") " Jul 06 19:37:26.881938 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:26.881890 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0f26e631-46f4-4b3e-8ee3-3a251e5c8305" (UID: "0f26e631-46f4-4b3e-8ee3-3a251e5c8305"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:26.883581 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:26.883554 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-kube-api-access-vl7fb" (OuterVolumeSpecName: "kube-api-access-vl7fb") pod "0f26e631-46f4-4b3e-8ee3-3a251e5c8305" (UID: "0f26e631-46f4-4b3e-8ee3-3a251e5c8305"). InnerVolumeSpecName "kube-api-access-vl7fb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:37:26.982256 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:26.982222 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-extensions-socket-volume\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:26.982256 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:26.982252 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl7fb\" (UniqueName: \"kubernetes.io/projected/0f26e631-46f4-4b3e-8ee3-3a251e5c8305-kube-api-access-vl7fb\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:27.517901 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:27.517865 2572 generic.go:356] "Generic (PLEG): container finished" podID="0f26e631-46f4-4b3e-8ee3-3a251e5c8305" containerID="51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09" exitCode=0 Jul 06 19:37:27.518310 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:27.517942 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" Jul 06 19:37:27.518310 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:27.517960 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" event={"ID":"0f26e631-46f4-4b3e-8ee3-3a251e5c8305","Type":"ContainerDied","Data":"51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09"} Jul 06 19:37:27.518310 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:27.518053 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld" event={"ID":"0f26e631-46f4-4b3e-8ee3-3a251e5c8305","Type":"ContainerDied","Data":"89e5b75570bc5546dfddd88265bc4a40a89e35049eb9c7e97872d9528e2b574e"} Jul 06 19:37:27.518310 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:27.518083 2572 scope.go:117] "RemoveContainer" containerID="51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09" Jul 06 19:37:27.527177 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:27.527161 2572 scope.go:117] "RemoveContainer" containerID="51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09" Jul 06 19:37:27.527462 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:37:27.527442 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09\": container with ID starting with 51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09 not found: ID does not exist" containerID="51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09" Jul 06 19:37:27.527503 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:27.527473 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09"} err="failed to get container status \"51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09\": rpc error: code = NotFound desc = could not find container \"51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09\": container with ID starting with 51e8c38307598e3d65730c77c1a0e061afef9de8b469394252a547dd558a7b09 not found: ID does not exist" Jul 06 19:37:27.551441 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:27.551411 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld"] Jul 06 19:37:27.557916 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:27.557878 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l6nld"] Jul 06 19:37:28.176170 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:28.176134 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f26e631-46f4-4b3e-8ee3-3a251e5c8305" path="/var/lib/kubelet/pods/0f26e631-46f4-4b3e-8ee3-3a251e5c8305/volumes" Jul 06 19:37:30.995735 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:30.995672 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:31.504556 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:31.504509 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:44.667813 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:44.667774 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:44.764130 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:44.764088 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:44.878418 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:44.878382 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:45.587479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.587441 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:45.646395 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.646350 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:45.751820 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.751789 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-p5pkx"] Jul 06 19:37:45.752225 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.752156 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f26e631-46f4-4b3e-8ee3-3a251e5c8305" containerName="manager" Jul 06 19:37:45.752225 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.752169 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f26e631-46f4-4b3e-8ee3-3a251e5c8305" containerName="manager" Jul 06 19:37:45.752301 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.752237 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f26e631-46f4-4b3e-8ee3-3a251e5c8305" containerName="manager" Jul 06 19:37:45.755281 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.755255 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p5pkx" Jul 06 19:37:45.757712 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.757684 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gkrqv\"" Jul 06 19:37:45.763599 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.763564 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p5pkx"] Jul 06 19:37:45.850769 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.850673 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c82w\" (UniqueName: \"kubernetes.io/projected/1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2-kube-api-access-8c82w\") pod \"authorino-75d556bd77-p5pkx\" (UID: \"1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2\") " pod="kuadrant-system/authorino-75d556bd77-p5pkx" Jul 06 19:37:45.952098 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.952056 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c82w\" (UniqueName: \"kubernetes.io/projected/1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2-kube-api-access-8c82w\") pod \"authorino-75d556bd77-p5pkx\" (UID: \"1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2\") " pod="kuadrant-system/authorino-75d556bd77-p5pkx" Jul 06 19:37:45.962248 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:45.962213 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c82w\" (UniqueName: \"kubernetes.io/projected/1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2-kube-api-access-8c82w\") pod \"authorino-75d556bd77-p5pkx\" (UID: \"1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2\") " pod="kuadrant-system/authorino-75d556bd77-p5pkx" Jul 06 19:37:46.065460 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:46.065428 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p5pkx" Jul 06 19:37:46.205745 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:46.205706 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p5pkx"] Jul 06 19:37:46.208433 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:37:46.208404 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e7d42e0_7e29_41b0_ac65_3a58cbd23ae2.slice/crio-8a19386da3caa6c54c85f9aff27388089509d9f8114ad241705e4454af092b8c WatchSource:0}: Error finding container 8a19386da3caa6c54c85f9aff27388089509d9f8114ad241705e4454af092b8c: Status 404 returned error can't find the container with id 8a19386da3caa6c54c85f9aff27388089509d9f8114ad241705e4454af092b8c Jul 06 19:37:46.610053 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:46.610024 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p5pkx" event={"ID":"1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2","Type":"ContainerStarted","Data":"8a19386da3caa6c54c85f9aff27388089509d9f8114ad241705e4454af092b8c"} Jul 06 19:37:50.630096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:50.630059 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p5pkx" event={"ID":"1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2","Type":"ContainerStarted","Data":"55e67909efa3726ab64d15e4c1ded016d6d9023f60b1bdc44bd40bf27b3d2faa"} Jul 06 19:37:50.646154 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:37:50.646093 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-p5pkx" podStartSLOduration=1.893263697 podStartE2EDuration="5.646078032s" totalImagesPullingTime="3.752814335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:37:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:37:50.645113529 +0000 UTC m=+781.150398567" watchObservedRunningTime="2026-07-06 19:37:50.646078032 +0000 UTC m=+781.151363069" Jul 06 19:38:04.758196 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:04.758154 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:16.783259 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:16.783175 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:16.991542 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:16.991506 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7dmsb"] Jul 06 19:38:16.994817 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:16.994796 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7dmsb" Jul 06 19:38:17.002141 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.002084 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7dmsb"] Jul 06 19:38:17.007185 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.007145 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:17.104977 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.104865 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww4nk\" (UniqueName: \"kubernetes.io/projected/c50392e0-96e8-45a9-9a0b-45c09219400c-kube-api-access-ww4nk\") pod \"authorino-5d4b67c446-7dmsb\" (UID: \"c50392e0-96e8-45a9-9a0b-45c09219400c\") " pod="kuadrant-system/authorino-5d4b67c446-7dmsb" Jul 06 19:38:17.206174 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.206137 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww4nk\" (UniqueName: \"kubernetes.io/projected/c50392e0-96e8-45a9-9a0b-45c09219400c-kube-api-access-ww4nk\") pod \"authorino-5d4b67c446-7dmsb\" (UID: \"c50392e0-96e8-45a9-9a0b-45c09219400c\") " pod="kuadrant-system/authorino-5d4b67c446-7dmsb" Jul 06 19:38:17.215646 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.215619 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww4nk\" (UniqueName: \"kubernetes.io/projected/c50392e0-96e8-45a9-9a0b-45c09219400c-kube-api-access-ww4nk\") pod \"authorino-5d4b67c446-7dmsb\" (UID: \"c50392e0-96e8-45a9-9a0b-45c09219400c\") " pod="kuadrant-system/authorino-5d4b67c446-7dmsb" Jul 06 19:38:17.229698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.229657 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7dmsb"] Jul 06 19:38:17.229985 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.229970 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7dmsb" Jul 06 19:38:17.256707 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.256672 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5648d4dc8f-t4ttl"] Jul 06 19:38:17.260444 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.260419 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5648d4dc8f-t4ttl" Jul 06 19:38:17.269164 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.269140 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5648d4dc8f-t4ttl"] Jul 06 19:38:17.307404 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.307367 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhhsr\" (UniqueName: \"kubernetes.io/projected/96ae9c9b-41ef-443d-a3f8-bd299518f842-kube-api-access-rhhsr\") pod \"authorino-5648d4dc8f-t4ttl\" (UID: \"96ae9c9b-41ef-443d-a3f8-bd299518f842\") " pod="kuadrant-system/authorino-5648d4dc8f-t4ttl" Jul 06 19:38:17.330566 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.330531 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5648d4dc8f-t4ttl"] Jul 06 19:38:17.330888 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:38:17.330861 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rhhsr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5648d4dc8f-t4ttl" podUID="96ae9c9b-41ef-443d-a3f8-bd299518f842" Jul 06 19:38:17.379776 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.379686 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7dmsb"] Jul 06 19:38:17.383076 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:38:17.383035 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc50392e0_96e8_45a9_9a0b_45c09219400c.slice/crio-8fa835015164060f87a61433d666f26f8cbeac72c69a959e47c2360789683eac WatchSource:0}: Error finding container 8fa835015164060f87a61433d666f26f8cbeac72c69a959e47c2360789683eac: Status 404 returned error can't find the container with id 8fa835015164060f87a61433d666f26f8cbeac72c69a959e47c2360789683eac Jul 06 19:38:17.408711 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.408677 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhhsr\" (UniqueName: \"kubernetes.io/projected/96ae9c9b-41ef-443d-a3f8-bd299518f842-kube-api-access-rhhsr\") pod \"authorino-5648d4dc8f-t4ttl\" (UID: \"96ae9c9b-41ef-443d-a3f8-bd299518f842\") " pod="kuadrant-system/authorino-5648d4dc8f-t4ttl" Jul 06 19:38:17.418190 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.418160 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhhsr\" (UniqueName: \"kubernetes.io/projected/96ae9c9b-41ef-443d-a3f8-bd299518f842-kube-api-access-rhhsr\") pod \"authorino-5648d4dc8f-t4ttl\" (UID: \"96ae9c9b-41ef-443d-a3f8-bd299518f842\") " pod="kuadrant-system/authorino-5648d4dc8f-t4ttl" Jul 06 19:38:17.751791 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.751760 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7dmsb" event={"ID":"c50392e0-96e8-45a9-9a0b-45c09219400c","Type":"ContainerStarted","Data":"eec6209eb18ca2ecde834b44ab483fdd69eda41f538681388157a6fae9f5cbf4"} Jul 06 19:38:17.751791 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.751778 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-7dmsb" podUID="c50392e0-96e8-45a9-9a0b-45c09219400c" containerName="authorino" containerID="cri-o://eec6209eb18ca2ecde834b44ab483fdd69eda41f538681388157a6fae9f5cbf4" gracePeriod=30 Jul 06 19:38:17.752000 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.751808 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7dmsb" event={"ID":"c50392e0-96e8-45a9-9a0b-45c09219400c","Type":"ContainerStarted","Data":"8fa835015164060f87a61433d666f26f8cbeac72c69a959e47c2360789683eac"} Jul 06 19:38:17.752000 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.751777 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5648d4dc8f-t4ttl" Jul 06 19:38:17.758923 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.758888 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5648d4dc8f-t4ttl" Jul 06 19:38:17.767542 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.767491 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-7dmsb" podStartSLOduration=1.458334166 podStartE2EDuration="1.767474982s" totalImagesPullingTime="309.140816ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:17.766613357 +0000 UTC m=+808.271898395" watchObservedRunningTime="2026-07-06 19:38:17.767474982 +0000 UTC m=+808.272760021" Jul 06 19:38:17.811828 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.811588 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhhsr\" (UniqueName: \"kubernetes.io/projected/96ae9c9b-41ef-443d-a3f8-bd299518f842-kube-api-access-rhhsr\") pod \"96ae9c9b-41ef-443d-a3f8-bd299518f842\" (UID: \"96ae9c9b-41ef-443d-a3f8-bd299518f842\") " Jul 06 19:38:17.814240 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.814210 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96ae9c9b-41ef-443d-a3f8-bd299518f842-kube-api-access-rhhsr" (OuterVolumeSpecName: "kube-api-access-rhhsr") pod "96ae9c9b-41ef-443d-a3f8-bd299518f842" (UID: "96ae9c9b-41ef-443d-a3f8-bd299518f842"). InnerVolumeSpecName "kube-api-access-rhhsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:38:17.912498 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:17.912465 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhhsr\" (UniqueName: \"kubernetes.io/projected/96ae9c9b-41ef-443d-a3f8-bd299518f842-kube-api-access-rhhsr\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:38:18.518003 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:18.517964 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p5pkx"] Jul 06 19:38:18.518223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:18.518194 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-p5pkx" podUID="1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2" containerName="authorino" containerID="cri-o://55e67909efa3726ab64d15e4c1ded016d6d9023f60b1bdc44bd40bf27b3d2faa" gracePeriod=30 Jul 06 19:38:18.763456 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:18.763415 2572 generic.go:356] "Generic (PLEG): container finished" podID="1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2" containerID="55e67909efa3726ab64d15e4c1ded016d6d9023f60b1bdc44bd40bf27b3d2faa" exitCode=0 Jul 06 19:38:18.763631 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:18.763482 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p5pkx" event={"ID":"1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2","Type":"ContainerDied","Data":"55e67909efa3726ab64d15e4c1ded016d6d9023f60b1bdc44bd40bf27b3d2faa"} Jul 06 19:38:18.763631 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:18.763555 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5648d4dc8f-t4ttl" Jul 06 19:38:18.780718 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:18.780696 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p5pkx" Jul 06 19:38:18.800030 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:18.799886 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5648d4dc8f-t4ttl"] Jul 06 19:38:18.801746 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:18.801714 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5648d4dc8f-t4ttl"] Jul 06 19:38:18.819738 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:18.819713 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8c82w\" (UniqueName: \"kubernetes.io/projected/1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2-kube-api-access-8c82w\") pod \"1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2\" (UID: \"1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2\") " Jul 06 19:38:18.822361 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:18.822333 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2-kube-api-access-8c82w" (OuterVolumeSpecName: "kube-api-access-8c82w") pod "1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2" (UID: "1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2"). InnerVolumeSpecName "kube-api-access-8c82w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:38:18.920929 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:18.920873 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8c82w\" (UniqueName: \"kubernetes.io/projected/1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2-kube-api-access-8c82w\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:38:19.768955 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:19.768917 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p5pkx" event={"ID":"1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2","Type":"ContainerDied","Data":"8a19386da3caa6c54c85f9aff27388089509d9f8114ad241705e4454af092b8c"} Jul 06 19:38:19.768955 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:19.768964 2572 scope.go:117] "RemoveContainer" containerID="55e67909efa3726ab64d15e4c1ded016d6d9023f60b1bdc44bd40bf27b3d2faa" Jul 06 19:38:19.769165 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:19.768932 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p5pkx" Jul 06 19:38:19.795646 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:19.795613 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p5pkx"] Jul 06 19:38:19.800057 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:19.800033 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p5pkx"] Jul 06 19:38:20.174525 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:20.174443 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2" path="/var/lib/kubelet/pods/1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2/volumes" Jul 06 19:38:20.174870 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:20.174760 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96ae9c9b-41ef-443d-a3f8-bd299518f842" path="/var/lib/kubelet/pods/96ae9c9b-41ef-443d-a3f8-bd299518f842/volumes" Jul 06 19:38:22.786829 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:22.786777 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:22.884507 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:22.884473 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:22.984673 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:22.984639 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:23.065058 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:23.064962 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:26.401488 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:26.401441 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:26.493347 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:26.493315 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:47.898821 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:47.898786 2572 generic.go:356] "Generic (PLEG): container finished" podID="c50392e0-96e8-45a9-9a0b-45c09219400c" containerID="eec6209eb18ca2ecde834b44ab483fdd69eda41f538681388157a6fae9f5cbf4" exitCode=137 Jul 06 19:38:47.899197 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:47.898859 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7dmsb" event={"ID":"c50392e0-96e8-45a9-9a0b-45c09219400c","Type":"ContainerDied","Data":"eec6209eb18ca2ecde834b44ab483fdd69eda41f538681388157a6fae9f5cbf4"} Jul 06 19:38:47.899197 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:47.898901 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7dmsb" event={"ID":"c50392e0-96e8-45a9-9a0b-45c09219400c","Type":"ContainerDied","Data":"8fa835015164060f87a61433d666f26f8cbeac72c69a959e47c2360789683eac"} Jul 06 19:38:47.899197 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:47.898933 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8fa835015164060f87a61433d666f26f8cbeac72c69a959e47c2360789683eac" Jul 06 19:38:47.910882 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:47.910862 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7dmsb" Jul 06 19:38:48.080990 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:48.080887 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww4nk\" (UniqueName: \"kubernetes.io/projected/c50392e0-96e8-45a9-9a0b-45c09219400c-kube-api-access-ww4nk\") pod \"c50392e0-96e8-45a9-9a0b-45c09219400c\" (UID: \"c50392e0-96e8-45a9-9a0b-45c09219400c\") " Jul 06 19:38:48.083058 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:48.083032 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c50392e0-96e8-45a9-9a0b-45c09219400c-kube-api-access-ww4nk" (OuterVolumeSpecName: "kube-api-access-ww4nk") pod "c50392e0-96e8-45a9-9a0b-45c09219400c" (UID: "c50392e0-96e8-45a9-9a0b-45c09219400c"). InnerVolumeSpecName "kube-api-access-ww4nk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:38:48.182370 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:48.182341 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww4nk\" (UniqueName: \"kubernetes.io/projected/c50392e0-96e8-45a9-9a0b-45c09219400c-kube-api-access-ww4nk\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:38:48.902557 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:48.902527 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7dmsb" Jul 06 19:38:48.918861 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:48.918824 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7dmsb"] Jul 06 19:38:48.920953 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:48.920901 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7dmsb"] Jul 06 19:38:50.175724 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:50.175690 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c50392e0-96e8-45a9-9a0b-45c09219400c" path="/var/lib/kubelet/pods/c50392e0-96e8-45a9-9a0b-45c09219400c/volumes" Jul 06 19:38:57.918755 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:57.918714 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:58.340036 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.340000 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:58.391652 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.391615 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-b6cbd65b-lhch9"] Jul 06 19:38:58.392068 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.392050 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2" containerName="authorino" Jul 06 19:38:58.392068 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.392070 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2" containerName="authorino" Jul 06 19:38:58.392222 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.392089 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c50392e0-96e8-45a9-9a0b-45c09219400c" containerName="authorino" Jul 06 19:38:58.392222 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.392097 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c50392e0-96e8-45a9-9a0b-45c09219400c" containerName="authorino" Jul 06 19:38:58.392333 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.392221 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e7d42e0-7e29-41b0-ac65-3a58cbd23ae2" containerName="authorino" Jul 06 19:38:58.392333 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.392236 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c50392e0-96e8-45a9-9a0b-45c09219400c" containerName="authorino" Jul 06 19:38:58.395519 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.395491 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:38:58.398383 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.398354 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 19:38:58.398650 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.398632 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-bt2cj\"" Jul 06 19:38:58.399960 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.399750 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-b6cbd65b-lhch9"] Jul 06 19:38:58.573733 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.573697 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d397eeb-06d2-4a41-9518-a974b9c64d03-maas-api-tls\") pod \"maas-api-b6cbd65b-lhch9\" (UID: \"1d397eeb-06d2-4a41-9518-a974b9c64d03\") " pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:38:58.573733 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.573733 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d397eeb-06d2-4a41-9518-a974b9c64d03-openshift-service-ca\") pod \"maas-api-b6cbd65b-lhch9\" (UID: \"1d397eeb-06d2-4a41-9518-a974b9c64d03\") " pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:38:58.573998 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.573831 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh7zr\" (UniqueName: \"kubernetes.io/projected/1d397eeb-06d2-4a41-9518-a974b9c64d03-kube-api-access-xh7zr\") pod \"maas-api-b6cbd65b-lhch9\" (UID: \"1d397eeb-06d2-4a41-9518-a974b9c64d03\") " pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:38:58.613123 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.613037 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:58.674314 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.674275 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d397eeb-06d2-4a41-9518-a974b9c64d03-maas-api-tls\") pod \"maas-api-b6cbd65b-lhch9\" (UID: \"1d397eeb-06d2-4a41-9518-a974b9c64d03\") " pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:38:58.674453 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.674318 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d397eeb-06d2-4a41-9518-a974b9c64d03-openshift-service-ca\") pod \"maas-api-b6cbd65b-lhch9\" (UID: \"1d397eeb-06d2-4a41-9518-a974b9c64d03\") " pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:38:58.674453 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.674361 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7zr\" (UniqueName: \"kubernetes.io/projected/1d397eeb-06d2-4a41-9518-a974b9c64d03-kube-api-access-xh7zr\") pod \"maas-api-b6cbd65b-lhch9\" (UID: \"1d397eeb-06d2-4a41-9518-a974b9c64d03\") " pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:38:58.675030 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.675005 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d397eeb-06d2-4a41-9518-a974b9c64d03-openshift-service-ca\") pod \"maas-api-b6cbd65b-lhch9\" (UID: \"1d397eeb-06d2-4a41-9518-a974b9c64d03\") " pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:38:58.676744 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.676715 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d397eeb-06d2-4a41-9518-a974b9c64d03-maas-api-tls\") pod \"maas-api-b6cbd65b-lhch9\" (UID: \"1d397eeb-06d2-4a41-9518-a974b9c64d03\") " pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:38:58.683366 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.683342 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh7zr\" (UniqueName: \"kubernetes.io/projected/1d397eeb-06d2-4a41-9518-a974b9c64d03-kube-api-access-xh7zr\") pod \"maas-api-b6cbd65b-lhch9\" (UID: \"1d397eeb-06d2-4a41-9518-a974b9c64d03\") " pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:38:58.709166 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.709130 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:38:58.860301 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.860266 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-b6cbd65b-lhch9"] Jul 06 19:38:58.862168 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:38:58.862138 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d397eeb_06d2_4a41_9518_a974b9c64d03.slice/crio-d3475280f2a0ad40052fda3bd7676f8e446c591ebc5738566136a47803f2aba0 WatchSource:0}: Error finding container d3475280f2a0ad40052fda3bd7676f8e446c591ebc5738566136a47803f2aba0: Status 404 returned error can't find the container with id d3475280f2a0ad40052fda3bd7676f8e446c591ebc5738566136a47803f2aba0 Jul 06 19:38:58.906169 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.906130 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:58.948972 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:58.948943 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-b6cbd65b-lhch9" event={"ID":"1d397eeb-06d2-4a41-9518-a974b9c64d03","Type":"ContainerStarted","Data":"d3475280f2a0ad40052fda3bd7676f8e446c591ebc5738566136a47803f2aba0"} Jul 06 19:38:59.002818 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.002778 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:59.143896 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.143819 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-fdtsv"] Jul 06 19:38:59.146422 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.146402 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" Jul 06 19:38:59.153149 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.153105 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9zchz\"" Jul 06 19:38:59.153378 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.153218 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 19:38:59.167290 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.167256 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-fdtsv"] Jul 06 19:38:59.179932 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.179886 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1405983-5015-401d-a880-cb4abbd175f8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-fdtsv\" (UID: \"a1405983-5015-401d-a880-cb4abbd175f8\") " pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" Jul 06 19:38:59.180089 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.179945 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5mzf\" (UniqueName: \"kubernetes.io/projected/a1405983-5015-401d-a880-cb4abbd175f8-kube-api-access-w5mzf\") pod \"payload-pre-processing-7964457696-fdtsv\" (UID: \"a1405983-5015-401d-a880-cb4abbd175f8\") " pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" Jul 06 19:38:59.281295 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.281259 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1405983-5015-401d-a880-cb4abbd175f8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-fdtsv\" (UID: \"a1405983-5015-401d-a880-cb4abbd175f8\") " pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" Jul 06 19:38:59.281295 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.281296 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mzf\" (UniqueName: \"kubernetes.io/projected/a1405983-5015-401d-a880-cb4abbd175f8-kube-api-access-w5mzf\") pod \"payload-pre-processing-7964457696-fdtsv\" (UID: \"a1405983-5015-401d-a880-cb4abbd175f8\") " pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" Jul 06 19:38:59.282075 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.282048 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1405983-5015-401d-a880-cb4abbd175f8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-fdtsv\" (UID: \"a1405983-5015-401d-a880-cb4abbd175f8\") " pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" Jul 06 19:38:59.291574 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.291539 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5mzf\" (UniqueName: \"kubernetes.io/projected/a1405983-5015-401d-a880-cb4abbd175f8-kube-api-access-w5mzf\") pod \"payload-pre-processing-7964457696-fdtsv\" (UID: \"a1405983-5015-401d-a880-cb4abbd175f8\") " pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" Jul 06 19:38:59.457208 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.457109 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" Jul 06 19:38:59.596629 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.596593 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-fdtsv"] Jul 06 19:38:59.598920 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:38:59.598867 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1405983_5015_401d_a880_cb4abbd175f8.slice/crio-6512a6bd0742d4eef209c60e453662be423de5d5f7b1553ffe3051a15d6c6382 WatchSource:0}: Error finding container 6512a6bd0742d4eef209c60e453662be423de5d5f7b1553ffe3051a15d6c6382: Status 404 returned error can't find the container with id 6512a6bd0742d4eef209c60e453662be423de5d5f7b1553ffe3051a15d6c6382 Jul 06 19:38:59.956596 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:38:59.956555 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" event={"ID":"a1405983-5015-401d-a880-cb4abbd175f8","Type":"ContainerStarted","Data":"6512a6bd0742d4eef209c60e453662be423de5d5f7b1553ffe3051a15d6c6382"} Jul 06 19:39:01.969237 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:01.969207 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-b6cbd65b-lhch9" event={"ID":"1d397eeb-06d2-4a41-9518-a974b9c64d03","Type":"ContainerStarted","Data":"5bd3adfabc7f065d989a830807cfaddeaa52effc4a931764ec39a5dca1190029"} Jul 06 19:39:01.969685 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:01.969305 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:39:01.970773 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:01.970751 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" event={"ID":"a1405983-5015-401d-a880-cb4abbd175f8","Type":"ContainerStarted","Data":"55538fdd9d48ba72c16c25bf30b4938c3f5edcf932e088d26ae6bca0327d8549"} Jul 06 19:39:01.970879 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:01.970803 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" Jul 06 19:39:01.987888 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:01.987840 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-b6cbd65b-lhch9" podStartSLOduration=1.10454174 podStartE2EDuration="3.987822659s" totalImagesPullingTime="2.883280919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:39:01.986228626 +0000 UTC m=+852.491513664" watchObservedRunningTime="2026-07-06 19:39:01.987822659 +0000 UTC m=+852.493107734" Jul 06 19:39:02.003896 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:02.003802 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" podStartSLOduration=0.858536375 podStartE2EDuration="3.003787374s" totalImagesPullingTime="2.145250999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:39:02.003224929 +0000 UTC m=+852.508509968" watchObservedRunningTime="2026-07-06 19:39:02.003787374 +0000 UTC m=+852.509072411" Jul 06 19:39:07.981984 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:07.981951 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-b6cbd65b-lhch9" Jul 06 19:39:09.201875 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:09.201827 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:09.294885 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:09.294848 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:12.978027 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:12.977974 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" podUID="a1405983-5015-401d-a880-cb4abbd175f8" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.40:9004: connect: connection refused" Jul 06 19:39:22.976780 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:22.976750 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-fdtsv" Jul 06 19:39:25.509988 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:25.509953 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:39.175825 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.175789 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2"] Jul 06 19:39:39.180783 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.180755 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.183158 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.183133 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pmg8n\"" Jul 06 19:39:39.183296 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.183133 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 19:39:39.183296 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.183133 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 19:39:39.183957 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.183942 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 19:39:39.187688 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.187658 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2"] Jul 06 19:39:39.213163 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.213133 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:39.310161 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.310130 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.310161 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.310162 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.310382 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.310186 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba17bcd2-f7a7-4966-a57a-4371830a6346-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.310382 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.310203 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhcrg\" (UniqueName: \"kubernetes.io/projected/ba17bcd2-f7a7-4966-a57a-4371830a6346-kube-api-access-vhcrg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.310382 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.310231 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.310382 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.310364 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.411181 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.411152 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.411382 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.411190 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.411382 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.411207 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.411382 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.411227 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba17bcd2-f7a7-4966-a57a-4371830a6346-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.411382 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.411252 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhcrg\" (UniqueName: \"kubernetes.io/projected/ba17bcd2-f7a7-4966-a57a-4371830a6346-kube-api-access-vhcrg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.411382 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.411279 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.411672 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.411622 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.411713 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.411685 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.411749 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.411716 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.413636 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.413614 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba17bcd2-f7a7-4966-a57a-4371830a6346-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.413951 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.413930 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba17bcd2-f7a7-4966-a57a-4371830a6346-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.421627 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.421601 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhcrg\" (UniqueName: \"kubernetes.io/projected/ba17bcd2-f7a7-4966-a57a-4371830a6346-kube-api-access-vhcrg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bxnf2\" (UID: \"ba17bcd2-f7a7-4966-a57a-4371830a6346\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.513596 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.513559 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:39.654238 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:39.654212 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2"] Jul 06 19:39:39.656739 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:39:39.656715 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba17bcd2_f7a7_4966_a57a_4371830a6346.slice/crio-4ebfdc3de78460cd6de11f1808ac6d5ebc98d8193b36b0e1e875735f6f14a663 WatchSource:0}: Error finding container 4ebfdc3de78460cd6de11f1808ac6d5ebc98d8193b36b0e1e875735f6f14a663: Status 404 returned error can't find the container with id 4ebfdc3de78460cd6de11f1808ac6d5ebc98d8193b36b0e1e875735f6f14a663 Jul 06 19:39:40.155186 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:40.155123 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" event={"ID":"ba17bcd2-f7a7-4966-a57a-4371830a6346","Type":"ContainerStarted","Data":"4ebfdc3de78460cd6de11f1808ac6d5ebc98d8193b36b0e1e875735f6f14a663"} Jul 06 19:39:41.161669 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:41.161626 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" event={"ID":"ba17bcd2-f7a7-4966-a57a-4371830a6346","Type":"ContainerStarted","Data":"cf4da370fc1b15eac5275f3c2bae712908bc4d933debba43d3298cf812a36c26"} Jul 06 19:39:41.180237 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:41.180187 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" podStartSLOduration=1.12975816 podStartE2EDuration="2.18017067s" totalImagesPullingTime="1.05041251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:39:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:39:41.179073329 +0000 UTC m=+891.684358380" watchObservedRunningTime="2026-07-06 19:39:41.18017067 +0000 UTC m=+891.685455708" Jul 06 19:39:49.514622 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:49.514536 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:49.514622 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:49.514600 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:49.527437 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:49.527412 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:49.964008 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:49.963978 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:39:49.964514 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:49.964494 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:39:49.968033 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:49.968013 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:39:49.968499 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:49.968481 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:39:50.212257 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:50.212226 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bxnf2" Jul 06 19:39:51.024056 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:39:51.024017 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:05.402844 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.402805 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8"] Jul 06 19:40:05.405991 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.405973 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.408401 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.408376 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 06 19:40:05.415442 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.415420 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8"] Jul 06 19:40:05.440350 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.440313 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.440350 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.440350 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.440609 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.440382 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.440609 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.440458 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2487f\" (UniqueName: \"kubernetes.io/projected/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-kube-api-access-2487f\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.440609 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.440514 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.440609 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.440553 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.443969 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.443936 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:05.541267 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.541227 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2487f\" (UniqueName: \"kubernetes.io/projected/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-kube-api-access-2487f\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.541475 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.541279 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.541475 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.541305 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.541475 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.541342 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.541475 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.541362 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.541475 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.541404 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.541811 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.541789 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.541858 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.541813 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.541858 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.541840 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.543812 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.543788 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.544148 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.544124 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.550102 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.550078 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2487f\" (UniqueName: \"kubernetes.io/projected/5386d35a-6af2-4cd1-affa-ed864ed8f9f9-kube-api-access-2487f\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8\" (UID: \"5386d35a-6af2-4cd1-affa-ed864ed8f9f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.718020 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.717979 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:05.860464 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.860373 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8"] Jul 06 19:40:05.862729 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:40:05.862695 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5386d35a_6af2_4cd1_affa_ed864ed8f9f9.slice/crio-13b8ff8265118c3840971763544b825654a0178f1e0ece678a8d5c728f5e77b7 WatchSource:0}: Error finding container 13b8ff8265118c3840971763544b825654a0178f1e0ece678a8d5c728f5e77b7: Status 404 returned error can't find the container with id 13b8ff8265118c3840971763544b825654a0178f1e0ece678a8d5c728f5e77b7 Jul 06 19:40:05.864369 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:05.864348 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:40:06.276802 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:06.276763 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" event={"ID":"5386d35a-6af2-4cd1-affa-ed864ed8f9f9","Type":"ContainerStarted","Data":"13b8ff8265118c3840971763544b825654a0178f1e0ece678a8d5c728f5e77b7"} Jul 06 19:40:07.283102 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:07.283063 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" event={"ID":"5386d35a-6af2-4cd1-affa-ed864ed8f9f9","Type":"ContainerStarted","Data":"b8256cd605c75f8ffb170b64d91f81b95dff87923a4666b9c820c99ce95ecd4d"} Jul 06 19:40:07.301200 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:07.301151 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" podStartSLOduration=1.667440584 podStartE2EDuration="2.301135526s" totalImagesPullingTime="633.694942ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:40:07.300237603 +0000 UTC m=+917.805522642" watchObservedRunningTime="2026-07-06 19:40:07.301135526 +0000 UTC m=+917.806420563" Jul 06 19:40:08.252475 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:08.252438 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:08.351056 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:08.351018 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:10.603948 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:10.603919 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:13.392143 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.392113 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8"] Jul 06 19:40:13.397522 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.397494 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.399982 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.399952 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 06 19:40:13.404823 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.404799 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8"] Jul 06 19:40:13.431645 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.431606 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:13.507264 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.507233 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.507264 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.507266 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.507479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.507329 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.507479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.507347 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.507479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.507412 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26490728-c426-46f3-99ec-d0d165ef0999-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.507479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.507464 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrdgx\" (UniqueName: \"kubernetes.io/projected/26490728-c426-46f3-99ec-d0d165ef0999-kube-api-access-hrdgx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.608699 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.608660 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.608699 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.608698 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.608991 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.608733 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26490728-c426-46f3-99ec-d0d165ef0999-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.608991 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.608771 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrdgx\" (UniqueName: \"kubernetes.io/projected/26490728-c426-46f3-99ec-d0d165ef0999-kube-api-access-hrdgx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.608991 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.608797 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.608991 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.608812 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.609372 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.609164 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.609372 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.609195 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.609372 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.609249 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.611120 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.611102 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26490728-c426-46f3-99ec-d0d165ef0999-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.611360 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.611343 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26490728-c426-46f3-99ec-d0d165ef0999-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.617084 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.617062 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrdgx\" (UniqueName: \"kubernetes.io/projected/26490728-c426-46f3-99ec-d0d165ef0999-kube-api-access-hrdgx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8\" (UID: \"26490728-c426-46f3-99ec-d0d165ef0999\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.709005 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.708879 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:13.854589 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:13.854548 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8"] Jul 06 19:40:13.857732 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:40:13.857703 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26490728_c426_46f3_99ec_d0d165ef0999.slice/crio-fcc4149473f3a781bc33d6759a8c61c18f4c9cbf0767d129cd7c3763478de5d7 WatchSource:0}: Error finding container fcc4149473f3a781bc33d6759a8c61c18f4c9cbf0767d129cd7c3763478de5d7: Status 404 returned error can't find the container with id fcc4149473f3a781bc33d6759a8c61c18f4c9cbf0767d129cd7c3763478de5d7 Jul 06 19:40:14.317540 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:14.317445 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" event={"ID":"26490728-c426-46f3-99ec-d0d165ef0999","Type":"ContainerStarted","Data":"88531a785b0e2f45c666b8b5f28a99cc8ca8f446494885c6214ec5412082dd9f"} Jul 06 19:40:14.317540 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:14.317491 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" event={"ID":"26490728-c426-46f3-99ec-d0d165ef0999","Type":"ContainerStarted","Data":"fcc4149473f3a781bc33d6759a8c61c18f4c9cbf0767d129cd7c3763478de5d7"} Jul 06 19:40:14.336414 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:14.336357 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" podStartSLOduration=1.130338316 podStartE2EDuration="1.336341868s" totalImagesPullingTime="206.003552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:40:14.334964438 +0000 UTC m=+924.840249478" watchObservedRunningTime="2026-07-06 19:40:14.336341868 +0000 UTC m=+924.841626906" Jul 06 19:40:15.718466 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:15.718414 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:15.718855 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:15.718489 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:15.731390 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:15.731365 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:16.339107 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:16.339073 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8" Jul 06 19:40:23.710048 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:23.709996 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:23.710502 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:23.710065 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:23.722671 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:23.722646 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:40:24.379228 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:40:24.379198 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8" Jul 06 19:41:38.593178 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:41:38.593136 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:38.695872 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:41:38.695831 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:39.722154 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:41:39.722120 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:39.823773 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:41:39.823732 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:48.802871 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:41:48.802829 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:03.468026 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:42:03.467989 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:10.296326 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:42:10.296291 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:21.327826 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:42:21.327790 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:50.284687 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:42:50.284647 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:51.616006 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:42:51.615979 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:12.381448 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:12.381407 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:15.251539 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:15.251505 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:33.727206 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:33.727168 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:33.823282 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:33.823246 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:33.938990 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:33.938953 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:39.819154 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:39.819111 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:39.883069 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:39.883031 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:39.967053 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:39.967007 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:40.069449 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:40.069360 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:40.173853 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:40.173817 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:40.564026 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:40.563990 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:46.140979 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:46.140944 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:46.200073 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:43:46.200032 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:44:12.177996 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:44:12.177896 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:44:12.278824 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:44:12.278785 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:44:49.992669 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:44:49.992640 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:44:49.993631 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:44:49.993607 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:44:49.996880 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:44:49.996850 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:44:49.997892 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:44:49.997872 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:44:50.039157 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:44:50.039129 2572 scope.go:117] "RemoveContainer" containerID="eec6209eb18ca2ecde834b44ab483fdd69eda41f538681388157a6fae9f5cbf4" Jul 06 19:47:45.623256 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:45.623219 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh"] Jul 06 19:47:45.623763 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:45.623412 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" podUID="3c0f207d-d493-452b-a771-c127bebe6843" containerName="manager" containerID="cri-o://260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8" gracePeriod=10 Jul 06 19:47:48.983867 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:48.983838 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:47:49.180829 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.180736 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-776h9\" (UniqueName: \"kubernetes.io/projected/3c0f207d-d493-452b-a771-c127bebe6843-kube-api-access-776h9\") pod \"3c0f207d-d493-452b-a771-c127bebe6843\" (UID: \"3c0f207d-d493-452b-a771-c127bebe6843\") " Jul 06 19:47:49.180829 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.180807 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c0f207d-d493-452b-a771-c127bebe6843-extensions-socket-volume\") pod \"3c0f207d-d493-452b-a771-c127bebe6843\" (UID: \"3c0f207d-d493-452b-a771-c127bebe6843\") " Jul 06 19:47:49.181235 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.181200 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0f207d-d493-452b-a771-c127bebe6843-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3c0f207d-d493-452b-a771-c127bebe6843" (UID: "3c0f207d-d493-452b-a771-c127bebe6843"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:47:49.182899 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.182867 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c0f207d-d493-452b-a771-c127bebe6843-kube-api-access-776h9" (OuterVolumeSpecName: "kube-api-access-776h9") pod "3c0f207d-d493-452b-a771-c127bebe6843" (UID: "3c0f207d-d493-452b-a771-c127bebe6843"). InnerVolumeSpecName "kube-api-access-776h9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:47:49.281428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.281389 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c0f207d-d493-452b-a771-c127bebe6843-extensions-socket-volume\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:47:49.281428 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.281421 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-776h9\" (UniqueName: \"kubernetes.io/projected/3c0f207d-d493-452b-a771-c127bebe6843-kube-api-access-776h9\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:47:49.495895 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.495859 2572 generic.go:356] "Generic (PLEG): container finished" podID="3c0f207d-d493-452b-a771-c127bebe6843" containerID="260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8" exitCode=0 Jul 06 19:47:49.496079 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.495930 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" Jul 06 19:47:49.496079 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.495946 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" event={"ID":"3c0f207d-d493-452b-a771-c127bebe6843","Type":"ContainerDied","Data":"260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8"} Jul 06 19:47:49.496079 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.496006 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh" event={"ID":"3c0f207d-d493-452b-a771-c127bebe6843","Type":"ContainerDied","Data":"5df48ac0c4524a6045476c881eb619465f7f6b57d5f4413402a7d552cf1ddcd2"} Jul 06 19:47:49.496079 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.496033 2572 scope.go:117] "RemoveContainer" containerID="260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8" Jul 06 19:47:49.506713 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.506692 2572 scope.go:117] "RemoveContainer" containerID="260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8" Jul 06 19:47:49.507027 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:47:49.507004 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8\": container with ID starting with 260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8 not found: ID does not exist" containerID="260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8" Jul 06 19:47:49.507085 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.507037 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8"} err="failed to get container status \"260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8\": rpc error: code = NotFound desc = could not find container \"260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8\": container with ID starting with 260a2c4b5063b6584b414e053c1fe99494a709b096dff6f34ed8b528ed4158a8 not found: ID does not exist" Jul 06 19:47:49.519120 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.519091 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh"] Jul 06 19:47:49.526346 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:49.526305 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hxhh"] Jul 06 19:47:50.175715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:47:50.175683 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c0f207d-d493-452b-a771-c127bebe6843" path="/var/lib/kubelet/pods/3c0f207d-d493-452b-a771-c127bebe6843/volumes" Jul 06 19:48:34.507950 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:34.507917 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:45.096407 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:45.096326 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:46.754356 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.754322 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p"] Jul 06 19:48:46.754745 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.754683 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c0f207d-d493-452b-a771-c127bebe6843" containerName="manager" Jul 06 19:48:46.754745 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.754694 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c0f207d-d493-452b-a771-c127bebe6843" containerName="manager" Jul 06 19:48:46.754825 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.754759 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c0f207d-d493-452b-a771-c127bebe6843" containerName="manager" Jul 06 19:48:46.757324 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.757307 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 19:48:46.759602 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.759581 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jt9tx\"" Jul 06 19:48:46.776922 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.776861 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p"] Jul 06 19:48:46.870384 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.870339 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdqzn\" (UniqueName: \"kubernetes.io/projected/bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa-kube-api-access-cdqzn\") pod \"kuadrant-operator-controller-manager-86878c769f-49k9p\" (UID: \"bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 19:48:46.870577 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.870452 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-49k9p\" (UID: \"bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 19:48:46.971882 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.971838 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdqzn\" (UniqueName: \"kubernetes.io/projected/bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa-kube-api-access-cdqzn\") pod \"kuadrant-operator-controller-manager-86878c769f-49k9p\" (UID: \"bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 19:48:46.972079 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.971933 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-49k9p\" (UID: \"bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 19:48:46.972292 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.972276 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-49k9p\" (UID: \"bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 19:48:46.983539 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:46.983504 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdqzn\" (UniqueName: \"kubernetes.io/projected/bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa-kube-api-access-cdqzn\") pod \"kuadrant-operator-controller-manager-86878c769f-49k9p\" (UID: \"bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 19:48:47.067880 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:47.067775 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 19:48:47.223608 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:47.223583 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p"] Jul 06 19:48:47.226392 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:48:47.226365 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd638e2_a4ec_4427_b1c1_5a43dc4aa7fa.slice/crio-74ac0a57b30995b793e4383f45ff68d3d01833408cb3bedfd28e2b2d9354cd29 WatchSource:0}: Error finding container 74ac0a57b30995b793e4383f45ff68d3d01833408cb3bedfd28e2b2d9354cd29: Status 404 returned error can't find the container with id 74ac0a57b30995b793e4383f45ff68d3d01833408cb3bedfd28e2b2d9354cd29 Jul 06 19:48:47.790041 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:47.789990 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" event={"ID":"bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa","Type":"ContainerStarted","Data":"6e2423185bad3453203495cf23b80680b604aacc070d5c6bab8f0eefeb132ee7"} Jul 06 19:48:47.790450 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:47.790069 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 19:48:47.790450 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:47.790091 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" event={"ID":"bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa","Type":"ContainerStarted","Data":"74ac0a57b30995b793e4383f45ff68d3d01833408cb3bedfd28e2b2d9354cd29"} Jul 06 19:48:47.827792 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:47.827731 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" podStartSLOduration=1.827715593 podStartE2EDuration="1.827715593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:47.825855249 +0000 UTC m=+1438.331140288" watchObservedRunningTime="2026-07-06 19:48:47.827715593 +0000 UTC m=+1438.333000630" Jul 06 19:48:58.798013 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:48:58.797977 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 19:49:50.023380 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:49:50.023349 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:49:50.024337 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:49:50.024310 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:49:50.027518 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:49:50.027498 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:49:50.028451 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:49:50.028430 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:51:34.712750 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:51:34.712705 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:09.497714 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:53:09.497673 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:53:11.298357 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:53:11.298317 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:50.052615 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:54:50.052586 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:54:50.053096 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:54:50.052994 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:54:50.056952 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:54:50.056894 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:54:50.057207 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:54:50.057184 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:54:51.242759 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:54:51.242727 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:54:56.537631 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:54:56.537589 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:04.799586 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:04.799548 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:09.898140 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:09.898103 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:12.745558 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:12.745521 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:12.842068 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:12.842033 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:13.136332 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:13.136232 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:19.043056 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:19.043020 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:19.144989 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:19.144949 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:19.251461 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:19.251428 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:19.356985 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:19.356852 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:20.423117 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.423084 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:20.486173 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.486135 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4"] Jul 06 19:55:20.490270 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.490240 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:20.492420 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.492393 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-9ce93acb-serving-cert\"" Jul 06 19:55:20.501732 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.501704 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4"] Jul 06 19:55:20.548859 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.548822 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:20.644134 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.644093 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7p5m\" (UniqueName: \"kubernetes.io/projected/6876487a-6659-4948-8475-2f75b714bd47-kube-api-access-r7p5m\") pod \"maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:20.644327 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.644221 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6876487a-6659-4948-8475-2f75b714bd47-maas-api-tls\") pod \"maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:20.644327 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.644249 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6876487a-6659-4948-8475-2f75b714bd47-openshift-service-ca\") pod \"maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:20.670191 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.670155 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:20.744919 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.744867 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7p5m\" (UniqueName: \"kubernetes.io/projected/6876487a-6659-4948-8475-2f75b714bd47-kube-api-access-r7p5m\") pod \"maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:20.745117 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.744979 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6876487a-6659-4948-8475-2f75b714bd47-maas-api-tls\") pod \"maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:20.745117 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.745004 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6876487a-6659-4948-8475-2f75b714bd47-openshift-service-ca\") pod \"maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:20.745117 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:55:20.745094 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-9ce93acb-serving-cert: secret "maas-api-e2e-ait-9ce93acb-serving-cert" not found Jul 06 19:55:20.745262 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:55:20.745178 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6876487a-6659-4948-8475-2f75b714bd47-maas-api-tls podName:6876487a-6659-4948-8475-2f75b714bd47 nodeName:}" failed. No retries permitted until 2026-07-06 19:55:21.245157154 +0000 UTC m=+1831.750442175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6876487a-6659-4948-8475-2f75b714bd47-maas-api-tls") pod "maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" (UID: "6876487a-6659-4948-8475-2f75b714bd47") : secret "maas-api-e2e-ait-9ce93acb-serving-cert" not found Jul 06 19:55:20.745867 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.745846 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6876487a-6659-4948-8475-2f75b714bd47-openshift-service-ca\") pod \"maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:20.754979 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.754944 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7p5m\" (UniqueName: \"kubernetes.io/projected/6876487a-6659-4948-8475-2f75b714bd47-kube-api-access-r7p5m\") pod \"maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:20.849918 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:20.849865 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:21.030603 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:21.030506 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:21.210919 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:21.210879 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:21.250074 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:21.250032 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6876487a-6659-4948-8475-2f75b714bd47-maas-api-tls\") pod \"maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:21.252484 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:21.252462 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6876487a-6659-4948-8475-2f75b714bd47-maas-api-tls\") pod \"maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:21.403236 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:21.403142 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:21.549241 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:21.549217 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4"] Jul 06 19:55:21.551493 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:55:21.551467 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6876487a_6659_4948_8475_2f75b714bd47.slice/crio-1e2746f853117f6bc00032dcc974557e3c4f3ac13dec6f69aa0bb58063b49cbc WatchSource:0}: Error finding container 1e2746f853117f6bc00032dcc974557e3c4f3ac13dec6f69aa0bb58063b49cbc: Status 404 returned error can't find the container with id 1e2746f853117f6bc00032dcc974557e3c4f3ac13dec6f69aa0bb58063b49cbc Jul 06 19:55:21.553281 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:21.553258 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:55:21.749970 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:21.749932 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" event={"ID":"6876487a-6659-4948-8475-2f75b714bd47","Type":"ContainerStarted","Data":"1e2746f853117f6bc00032dcc974557e3c4f3ac13dec6f69aa0bb58063b49cbc"} Jul 06 19:55:22.756806 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:22.756766 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" event={"ID":"6876487a-6659-4948-8475-2f75b714bd47","Type":"ContainerStarted","Data":"342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896"} Jul 06 19:55:22.757250 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:22.756931 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:22.779013 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:22.778960 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" podStartSLOduration=2.111103702 podStartE2EDuration="2.778943473s" totalImagesPullingTime="667.839771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:22.777267843 +0000 UTC m=+1833.282552881" watchObservedRunningTime="2026-07-06 19:55:22.778943473 +0000 UTC m=+1833.284228522" Jul 06 19:55:23.018305 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:23.018198 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:25.498278 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:25.498238 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:25.706503 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:25.706466 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:25.806827 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:25.806731 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:25.900035 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:25.900002 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4"] Jul 06 19:55:25.900212 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:25.900185 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" podUID="6876487a-6659-4948-8475-2f75b714bd47" containerName="maas-api" containerID="cri-o://342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896" gracePeriod=30 Jul 06 19:55:25.929241 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:25.929212 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:26.159127 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.159096 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:26.192921 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.192880 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6876487a-6659-4948-8475-2f75b714bd47-openshift-service-ca\") pod \"6876487a-6659-4948-8475-2f75b714bd47\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " Jul 06 19:55:26.193081 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.192989 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7p5m\" (UniqueName: \"kubernetes.io/projected/6876487a-6659-4948-8475-2f75b714bd47-kube-api-access-r7p5m\") pod \"6876487a-6659-4948-8475-2f75b714bd47\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " Jul 06 19:55:26.193142 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.193093 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6876487a-6659-4948-8475-2f75b714bd47-maas-api-tls\") pod \"6876487a-6659-4948-8475-2f75b714bd47\" (UID: \"6876487a-6659-4948-8475-2f75b714bd47\") " Jul 06 19:55:26.193302 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.193275 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6876487a-6659-4948-8475-2f75b714bd47-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6876487a-6659-4948-8475-2f75b714bd47" (UID: "6876487a-6659-4948-8475-2f75b714bd47"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:55:26.193383 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.193362 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6876487a-6659-4948-8475-2f75b714bd47-openshift-service-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:55:26.195271 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.195242 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6876487a-6659-4948-8475-2f75b714bd47-kube-api-access-r7p5m" (OuterVolumeSpecName: "kube-api-access-r7p5m") pod "6876487a-6659-4948-8475-2f75b714bd47" (UID: "6876487a-6659-4948-8475-2f75b714bd47"). InnerVolumeSpecName "kube-api-access-r7p5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:55:26.195386 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.195360 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6876487a-6659-4948-8475-2f75b714bd47-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6876487a-6659-4948-8475-2f75b714bd47" (UID: "6876487a-6659-4948-8475-2f75b714bd47"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:55:26.293785 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.293742 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6876487a-6659-4948-8475-2f75b714bd47-maas-api-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:55:26.293785 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.293782 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7p5m\" (UniqueName: \"kubernetes.io/projected/6876487a-6659-4948-8475-2f75b714bd47-kube-api-access-r7p5m\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:55:26.777221 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.777180 2572 generic.go:356] "Generic (PLEG): container finished" podID="6876487a-6659-4948-8475-2f75b714bd47" containerID="342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896" exitCode=0 Jul 06 19:55:26.777221 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.777225 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" Jul 06 19:55:26.777683 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.777264 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" event={"ID":"6876487a-6659-4948-8475-2f75b714bd47","Type":"ContainerDied","Data":"342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896"} Jul 06 19:55:26.777683 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.777323 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4" event={"ID":"6876487a-6659-4948-8475-2f75b714bd47","Type":"ContainerDied","Data":"1e2746f853117f6bc00032dcc974557e3c4f3ac13dec6f69aa0bb58063b49cbc"} Jul 06 19:55:26.777683 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.777347 2572 scope.go:117] "RemoveContainer" containerID="342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896" Jul 06 19:55:26.786609 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.786584 2572 scope.go:117] "RemoveContainer" containerID="342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896" Jul 06 19:55:26.786890 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:55:26.786873 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896\": container with ID starting with 342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896 not found: ID does not exist" containerID="342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896" Jul 06 19:55:26.786970 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.786901 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896"} err="failed to get container status \"342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896\": rpc error: code = NotFound desc = could not find container \"342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896\": container with ID starting with 342cf80435d573f85f9fd827e2e58a83a88599b914046df1b260434c1b987896 not found: ID does not exist" Jul 06 19:55:26.804008 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.803971 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4"] Jul 06 19:55:26.809828 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:26.809801 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-9ce93acb-d96c7fdbd-sdct4"] Jul 06 19:55:27.392804 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:27.392764 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:28.175599 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:28.175558 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6876487a-6659-4948-8475-2f75b714bd47" path="/var/lib/kubelet/pods/6876487a-6659-4948-8475-2f75b714bd47/volumes" Jul 06 19:55:31.850347 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:31.850309 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:31.956103 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:31.956066 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:32.104287 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:32.104197 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:32.205431 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:32.205397 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:33.279202 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.279154 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:33.333725 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.333691 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v"] Jul 06 19:55:33.334139 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.334120 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6876487a-6659-4948-8475-2f75b714bd47" containerName="maas-api" Jul 06 19:55:33.334139 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.334136 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6876487a-6659-4948-8475-2f75b714bd47" containerName="maas-api" Jul 06 19:55:33.334332 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.334218 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6876487a-6659-4948-8475-2f75b714bd47" containerName="maas-api" Jul 06 19:55:33.339361 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.339328 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:33.341672 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.341642 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-6dbb1fec-serving-cert\"" Jul 06 19:55:33.349821 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.349787 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v"] Jul 06 19:55:33.366918 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.366870 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:33.452681 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.452648 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hncvs\" (UniqueName: \"kubernetes.io/projected/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-kube-api-access-hncvs\") pod \"maas-api-e2e-ait-6dbb1fec-6f97695854-x749v\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:33.452930 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.452704 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-maas-api-tls\") pod \"maas-api-e2e-ait-6dbb1fec-6f97695854-x749v\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:33.452930 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.452783 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-openshift-service-ca\") pod \"maas-api-e2e-ait-6dbb1fec-6f97695854-x749v\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:33.453611 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.453577 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:33.554236 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.554142 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hncvs\" (UniqueName: \"kubernetes.io/projected/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-kube-api-access-hncvs\") pod \"maas-api-e2e-ait-6dbb1fec-6f97695854-x749v\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:33.554236 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.554199 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-maas-api-tls\") pod \"maas-api-e2e-ait-6dbb1fec-6f97695854-x749v\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:33.554469 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.554241 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-openshift-service-ca\") pod \"maas-api-e2e-ait-6dbb1fec-6f97695854-x749v\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:33.554985 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.554964 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-openshift-service-ca\") pod \"maas-api-e2e-ait-6dbb1fec-6f97695854-x749v\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:33.556474 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.556446 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:33.557158 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.557135 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-maas-api-tls\") pod \"maas-api-e2e-ait-6dbb1fec-6f97695854-x749v\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:33.563227 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.563189 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hncvs\" (UniqueName: \"kubernetes.io/projected/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-kube-api-access-hncvs\") pod \"maas-api-e2e-ait-6dbb1fec-6f97695854-x749v\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:33.655399 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.655351 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:33.816377 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.816351 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v"] Jul 06 19:55:33.819850 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:55:33.819821 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0b79a2_4bab_4347_a25d_af2ca4c4c2a5.slice/crio-bf052043c92187ebc8fac01dda3c1cec3d08241cc276db17dd326a85f2afeb76 WatchSource:0}: Error finding container bf052043c92187ebc8fac01dda3c1cec3d08241cc276db17dd326a85f2afeb76: Status 404 returned error can't find the container with id bf052043c92187ebc8fac01dda3c1cec3d08241cc276db17dd326a85f2afeb76 Jul 06 19:55:33.856939 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.856877 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:33.956891 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:33.956854 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:34.827355 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:34.827313 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" event={"ID":"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5","Type":"ContainerStarted","Data":"14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889"} Jul 06 19:55:34.827767 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:34.827371 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" event={"ID":"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5","Type":"ContainerStarted","Data":"bf052043c92187ebc8fac01dda3c1cec3d08241cc276db17dd326a85f2afeb76"} Jul 06 19:55:34.827767 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:34.827412 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:34.848719 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:34.848656 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" podStartSLOduration=1.441821041 podStartE2EDuration="1.848630961s" totalImagesPullingTime="406.80992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:34.84704724 +0000 UTC m=+1845.352332279" watchObservedRunningTime="2026-07-06 19:55:34.848630961 +0000 UTC m=+1845.353916001" Jul 06 19:55:35.674668 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:35.674549 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:38.466198 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:38.466148 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:38.784159 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:38.784068 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v"] Jul 06 19:55:38.784367 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:38.784321 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" podUID="bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5" containerName="maas-api" containerID="cri-o://14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889" gracePeriod=30 Jul 06 19:55:38.839793 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:38.839753 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:39.046607 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.046540 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:39.102294 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.102260 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hncvs\" (UniqueName: \"kubernetes.io/projected/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-kube-api-access-hncvs\") pod \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " Jul 06 19:55:39.102464 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.102311 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-maas-api-tls\") pod \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " Jul 06 19:55:39.102464 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.102428 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-openshift-service-ca\") pod \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\" (UID: \"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5\") " Jul 06 19:55:39.102861 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.102828 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5" (UID: "bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:55:39.104468 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.104439 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-kube-api-access-hncvs" (OuterVolumeSpecName: "kube-api-access-hncvs") pod "bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5" (UID: "bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5"). InnerVolumeSpecName "kube-api-access-hncvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:55:39.104625 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.104599 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5" (UID: "bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:55:39.203731 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.203679 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-openshift-service-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:55:39.203731 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.203723 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hncvs\" (UniqueName: \"kubernetes.io/projected/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-kube-api-access-hncvs\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:55:39.203731 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.203733 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5-maas-api-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:55:39.855098 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.855066 2572 generic.go:356] "Generic (PLEG): container finished" podID="bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5" containerID="14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889" exitCode=0 Jul 06 19:55:39.855481 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.855117 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" Jul 06 19:55:39.855481 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.855154 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" event={"ID":"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5","Type":"ContainerDied","Data":"14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889"} Jul 06 19:55:39.855481 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.855211 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v" event={"ID":"bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5","Type":"ContainerDied","Data":"bf052043c92187ebc8fac01dda3c1cec3d08241cc276db17dd326a85f2afeb76"} Jul 06 19:55:39.855481 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.855239 2572 scope.go:117] "RemoveContainer" containerID="14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889" Jul 06 19:55:39.864900 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.864884 2572 scope.go:117] "RemoveContainer" containerID="14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889" Jul 06 19:55:39.865206 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:55:39.865189 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889\": container with ID starting with 14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889 not found: ID does not exist" containerID="14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889" Jul 06 19:55:39.865269 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.865218 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889"} err="failed to get container status \"14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889\": rpc error: code = NotFound desc = could not find container \"14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889\": container with ID starting with 14e969d5f8e4e8ab43a0c8f709a15ef5d38d6c31140ada31d3ac330c1cc69889 not found: ID does not exist" Jul 06 19:55:39.878017 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.877989 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v"] Jul 06 19:55:39.882364 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:39.882339 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-6dbb1fec-6f97695854-x749v"] Jul 06 19:55:40.176623 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:40.176531 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5" path="/var/lib/kubelet/pods/bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5/volumes" Jul 06 19:55:41.153223 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:41.153179 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:44.511536 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:44.511498 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:44.624205 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:44.624164 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:46.670170 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:46.670137 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:51.049965 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:51.049929 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:51.125649 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:51.125607 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:51.318307 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:51.318222 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:51.418338 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:51.418306 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:52.468023 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.467981 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:52.567194 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.567157 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:52.577292 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.577257 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l"] Jul 06 19:55:52.577698 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.577681 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5" containerName="maas-api" Jul 06 19:55:52.577801 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.577701 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5" containerName="maas-api" Jul 06 19:55:52.577801 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.577798 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0b79a2-4bab-4347-a25d-af2ca4c4c2a5" containerName="maas-api" Jul 06 19:55:52.582537 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.582513 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:52.587148 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.587127 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-584eb001-serving-cert\"" Jul 06 19:55:52.600035 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.600004 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l"] Jul 06 19:55:52.664618 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.664582 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:52.732785 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.732684 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/813564ac-6b09-42ad-85b0-9b161a77a8e7-maas-api-tls\") pod \"maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:52.732785 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.732728 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfrzm\" (UniqueName: \"kubernetes.io/projected/813564ac-6b09-42ad-85b0-9b161a77a8e7-kube-api-access-mfrzm\") pod \"maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:52.733033 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.732823 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/813564ac-6b09-42ad-85b0-9b161a77a8e7-openshift-service-ca\") pod \"maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:52.764710 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.764670 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:52.833966 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.833884 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/813564ac-6b09-42ad-85b0-9b161a77a8e7-maas-api-tls\") pod \"maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:52.834144 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.833980 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfrzm\" (UniqueName: \"kubernetes.io/projected/813564ac-6b09-42ad-85b0-9b161a77a8e7-kube-api-access-mfrzm\") pod \"maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:52.834144 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.834044 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/813564ac-6b09-42ad-85b0-9b161a77a8e7-openshift-service-ca\") pod \"maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:52.834766 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.834735 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/813564ac-6b09-42ad-85b0-9b161a77a8e7-openshift-service-ca\") pod \"maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:52.836766 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.836743 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/813564ac-6b09-42ad-85b0-9b161a77a8e7-maas-api-tls\") pod \"maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:52.844949 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.844865 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfrzm\" (UniqueName: \"kubernetes.io/projected/813564ac-6b09-42ad-85b0-9b161a77a8e7-kube-api-access-mfrzm\") pod \"maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:52.893481 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:52.893443 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:53.040776 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:53.040744 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l"] Jul 06 19:55:53.044225 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:55:53.044195 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod813564ac_6b09_42ad_85b0_9b161a77a8e7.slice/crio-b315f10e22ea74fca7eff5fcdf06472939ac3c5ee449b7e48abfcfc4c33ec422 WatchSource:0}: Error finding container b315f10e22ea74fca7eff5fcdf06472939ac3c5ee449b7e48abfcfc4c33ec422: Status 404 returned error can't find the container with id b315f10e22ea74fca7eff5fcdf06472939ac3c5ee449b7e48abfcfc4c33ec422 Jul 06 19:55:53.304465 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:53.304361 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:53.405290 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:53.405260 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:53.924055 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:53.924017 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" event={"ID":"813564ac-6b09-42ad-85b0-9b161a77a8e7","Type":"ContainerStarted","Data":"5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598"} Jul 06 19:55:53.924465 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:53.924074 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" event={"ID":"813564ac-6b09-42ad-85b0-9b161a77a8e7","Type":"ContainerStarted","Data":"b315f10e22ea74fca7eff5fcdf06472939ac3c5ee449b7e48abfcfc4c33ec422"} Jul 06 19:55:53.924465 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:53.924116 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:53.943510 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:53.943453 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" podStartSLOduration=1.535173033 podStartE2EDuration="1.943434244s" totalImagesPullingTime="408.261211ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:53.941228284 +0000 UTC m=+1864.446513334" watchObservedRunningTime="2026-07-06 19:55:53.943434244 +0000 UTC m=+1864.448719282" Jul 06 19:55:54.495171 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:54.495132 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:56.921717 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:56.921682 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:57.150999 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.150953 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:57.249566 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.249531 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:57.455292 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.455161 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l"] Jul 06 19:55:57.455490 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.455398 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" podUID="813564ac-6b09-42ad-85b0-9b161a77a8e7" containerName="maas-api" containerID="cri-o://5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598" gracePeriod=30 Jul 06 19:55:57.483960 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.483878 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:55:57.708715 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.708688 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:57.780208 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.780173 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/813564ac-6b09-42ad-85b0-9b161a77a8e7-maas-api-tls\") pod \"813564ac-6b09-42ad-85b0-9b161a77a8e7\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " Jul 06 19:55:57.780412 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.780241 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/813564ac-6b09-42ad-85b0-9b161a77a8e7-openshift-service-ca\") pod \"813564ac-6b09-42ad-85b0-9b161a77a8e7\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " Jul 06 19:55:57.780412 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.780398 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfrzm\" (UniqueName: \"kubernetes.io/projected/813564ac-6b09-42ad-85b0-9b161a77a8e7-kube-api-access-mfrzm\") pod \"813564ac-6b09-42ad-85b0-9b161a77a8e7\" (UID: \"813564ac-6b09-42ad-85b0-9b161a77a8e7\") " Jul 06 19:55:57.780655 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.780627 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813564ac-6b09-42ad-85b0-9b161a77a8e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "813564ac-6b09-42ad-85b0-9b161a77a8e7" (UID: "813564ac-6b09-42ad-85b0-9b161a77a8e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:55:57.782426 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.782403 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/813564ac-6b09-42ad-85b0-9b161a77a8e7-kube-api-access-mfrzm" (OuterVolumeSpecName: "kube-api-access-mfrzm") pod "813564ac-6b09-42ad-85b0-9b161a77a8e7" (UID: "813564ac-6b09-42ad-85b0-9b161a77a8e7"). InnerVolumeSpecName "kube-api-access-mfrzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:55:57.782525 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.782483 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/813564ac-6b09-42ad-85b0-9b161a77a8e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "813564ac-6b09-42ad-85b0-9b161a77a8e7" (UID: "813564ac-6b09-42ad-85b0-9b161a77a8e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:55:57.881797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.881694 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfrzm\" (UniqueName: \"kubernetes.io/projected/813564ac-6b09-42ad-85b0-9b161a77a8e7-kube-api-access-mfrzm\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:55:57.881797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.881728 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/813564ac-6b09-42ad-85b0-9b161a77a8e7-maas-api-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:55:57.881797 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.881741 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/813564ac-6b09-42ad-85b0-9b161a77a8e7-openshift-service-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:55:57.946532 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.946491 2572 generic.go:356] "Generic (PLEG): container finished" podID="813564ac-6b09-42ad-85b0-9b161a77a8e7" containerID="5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598" exitCode=0 Jul 06 19:55:57.946532 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.946535 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" event={"ID":"813564ac-6b09-42ad-85b0-9b161a77a8e7","Type":"ContainerDied","Data":"5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598"} Jul 06 19:55:57.947183 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.946539 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" Jul 06 19:55:57.947183 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.946579 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l" event={"ID":"813564ac-6b09-42ad-85b0-9b161a77a8e7","Type":"ContainerDied","Data":"b315f10e22ea74fca7eff5fcdf06472939ac3c5ee449b7e48abfcfc4c33ec422"} Jul 06 19:55:57.947183 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.946601 2572 scope.go:117] "RemoveContainer" containerID="5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598" Jul 06 19:55:57.955750 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.955731 2572 scope.go:117] "RemoveContainer" containerID="5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598" Jul 06 19:55:57.956043 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:55:57.956025 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598\": container with ID starting with 5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598 not found: ID does not exist" containerID="5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598" Jul 06 19:55:57.956106 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.956049 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598"} err="failed to get container status \"5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598\": rpc error: code = NotFound desc = could not find container \"5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598\": container with ID starting with 5fb2c4f2d53fb0640f4f7cf9d6a575e9ffa06eedf241192638d7362531811598 not found: ID does not exist" Jul 06 19:55:57.975422 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.975383 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l"] Jul 06 19:55:57.980637 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:57.980608 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-584eb001-7458cbf4d9-kng8l"] Jul 06 19:55:58.176364 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:58.176273 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="813564ac-6b09-42ad-85b0-9b161a77a8e7" path="/var/lib/kubelet/pods/813564ac-6b09-42ad-85b0-9b161a77a8e7/volumes" Jul 06 19:55:58.714497 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:55:58.714452 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:03.457082 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:03.457043 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:03.561994 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:03.561964 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:03.935452 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:03.935403 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:04.042733 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:04.042683 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:04.147816 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:04.147766 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:05.581859 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:05.581796 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:11.529185 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:11.529148 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:11.632411 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:11.632373 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:11.730339 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:11.730300 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:11.832240 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:11.832158 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:12.255350 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:12.255319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:38.211521 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:38.211478 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:44.402149 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:44.402111 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:44.462992 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:44.462959 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:44.568309 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:44.568266 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:56:44.659077 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:56:44.658985 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:00.524013 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:00.523969 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:06.488672 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:06.488637 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:06.563391 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:06.563357 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:06.652539 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:06.652500 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:06.750931 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:06.750829 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:17.208007 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:17.207956 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:18.004379 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:18.004342 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:19.298203 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:19.298142 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:19.446155 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:19.446124 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:23.973458 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:23.973407 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:24.089171 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:24.089133 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:24.177433 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:24.177392 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:24.277573 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:24.277491 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:25.005227 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:25.005188 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:25.106297 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:25.106257 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:25.214700 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:25.214667 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:26.921135 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:26.921103 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:33.390466 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:33.390427 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:33.463559 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:33.463517 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:34.070825 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:34.070779 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:40.312271 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:40.312224 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:40.384041 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:40.384009 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:40.577461 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:40.577355 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:40.902499 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:40.902401 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:41.005788 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:41.005756 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:44.267279 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.267195 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:44.304380 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.304342 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb"] Jul 06 19:57:44.304733 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.304721 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="813564ac-6b09-42ad-85b0-9b161a77a8e7" containerName="maas-api" Jul 06 19:57:44.304733 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.304734 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="813564ac-6b09-42ad-85b0-9b161a77a8e7" containerName="maas-api" Jul 06 19:57:44.304849 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.304815 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="813564ac-6b09-42ad-85b0-9b161a77a8e7" containerName="maas-api" Jul 06 19:57:44.308136 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.308115 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:44.310531 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.310507 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-a04b5bb9-serving-cert\"" Jul 06 19:57:44.321402 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.321374 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb"] Jul 06 19:57:44.347773 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.347737 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:44.468749 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.468713 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-974jl\" (UniqueName: \"kubernetes.io/projected/c6d338bd-f164-45a8-ba86-ef468ff21813-kube-api-access-974jl\") pod \"maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:44.468953 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.468801 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6d338bd-f164-45a8-ba86-ef468ff21813-maas-api-tls\") pod \"maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:44.468953 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.468855 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d338bd-f164-45a8-ba86-ef468ff21813-openshift-service-ca\") pod \"maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:44.569790 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.569689 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6d338bd-f164-45a8-ba86-ef468ff21813-maas-api-tls\") pod \"maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:44.569790 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.569755 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d338bd-f164-45a8-ba86-ef468ff21813-openshift-service-ca\") pod \"maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:44.570073 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.569829 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-974jl\" (UniqueName: \"kubernetes.io/projected/c6d338bd-f164-45a8-ba86-ef468ff21813-kube-api-access-974jl\") pod \"maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:44.570539 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.570504 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d338bd-f164-45a8-ba86-ef468ff21813-openshift-service-ca\") pod \"maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:44.572266 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.572243 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6d338bd-f164-45a8-ba86-ef468ff21813-maas-api-tls\") pod \"maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:44.583175 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.583151 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-974jl\" (UniqueName: \"kubernetes.io/projected/c6d338bd-f164-45a8-ba86-ef468ff21813-kube-api-access-974jl\") pod \"maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:44.609259 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.609218 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:44.619154 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.619122 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:44.768478 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.768430 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb"] Jul 06 19:57:44.770716 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:57:44.770678 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d338bd_f164_45a8_ba86_ef468ff21813.slice/crio-4884762e2e21fb4653948dac881d4686e3b7b23c708e939d831a5e3ca6bcd342 WatchSource:0}: Error finding container 4884762e2e21fb4653948dac881d4686e3b7b23c708e939d831a5e3ca6bcd342: Status 404 returned error can't find the container with id 4884762e2e21fb4653948dac881d4686e3b7b23c708e939d831a5e3ca6bcd342 Jul 06 19:57:44.937047 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:44.936940 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:45.039246 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:45.039210 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:45.477348 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:45.477313 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" event={"ID":"c6d338bd-f164-45a8-ba86-ef468ff21813","Type":"ContainerStarted","Data":"4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7"} Jul 06 19:57:45.477744 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:45.477366 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" event={"ID":"c6d338bd-f164-45a8-ba86-ef468ff21813","Type":"ContainerStarted","Data":"4884762e2e21fb4653948dac881d4686e3b7b23c708e939d831a5e3ca6bcd342"} Jul 06 19:57:45.477744 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:45.477391 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:57:45.497805 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:45.497745 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" podStartSLOduration=1.011990479 podStartE2EDuration="1.497724753s" totalImagesPullingTime="485.734274ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:57:45.496418659 +0000 UTC m=+1976.001703698" watchObservedRunningTime="2026-07-06 19:57:45.497724753 +0000 UTC m=+1976.003009795" Jul 06 19:57:48.758455 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:48.758413 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:49.393778 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:49.393724 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:49.529842 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:49.529809 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:49.833154 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:49.833111 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:57:51.488193 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:57:51.488163 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:58:13.918499 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:13.918466 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:14.008277 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:14.008242 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:14.486814 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:14.486769 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:14.892688 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:14.892589 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:14.894026 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:14.893996 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb"] Jul 06 19:58:14.894250 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:14.894228 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" podUID="c6d338bd-f164-45a8-ba86-ef468ff21813" containerName="maas-api" containerID="cri-o://4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7" gracePeriod=30 Jul 06 19:58:14.984367 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:14.984328 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:15.152268 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.152194 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:58:15.246274 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.246237 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6d338bd-f164-45a8-ba86-ef468ff21813-maas-api-tls\") pod \"c6d338bd-f164-45a8-ba86-ef468ff21813\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " Jul 06 19:58:15.246472 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.246316 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-974jl\" (UniqueName: \"kubernetes.io/projected/c6d338bd-f164-45a8-ba86-ef468ff21813-kube-api-access-974jl\") pod \"c6d338bd-f164-45a8-ba86-ef468ff21813\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " Jul 06 19:58:15.246472 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.246387 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d338bd-f164-45a8-ba86-ef468ff21813-openshift-service-ca\") pod \"c6d338bd-f164-45a8-ba86-ef468ff21813\" (UID: \"c6d338bd-f164-45a8-ba86-ef468ff21813\") " Jul 06 19:58:15.246793 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.246755 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d338bd-f164-45a8-ba86-ef468ff21813-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c6d338bd-f164-45a8-ba86-ef468ff21813" (UID: "c6d338bd-f164-45a8-ba86-ef468ff21813"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:58:15.248493 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.248466 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d338bd-f164-45a8-ba86-ef468ff21813-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c6d338bd-f164-45a8-ba86-ef468ff21813" (UID: "c6d338bd-f164-45a8-ba86-ef468ff21813"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:58:15.248582 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.248514 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d338bd-f164-45a8-ba86-ef468ff21813-kube-api-access-974jl" (OuterVolumeSpecName: "kube-api-access-974jl") pod "c6d338bd-f164-45a8-ba86-ef468ff21813" (UID: "c6d338bd-f164-45a8-ba86-ef468ff21813"). InnerVolumeSpecName "kube-api-access-974jl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:58:15.347747 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.347706 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-974jl\" (UniqueName: \"kubernetes.io/projected/c6d338bd-f164-45a8-ba86-ef468ff21813-kube-api-access-974jl\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:58:15.347747 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.347740 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d338bd-f164-45a8-ba86-ef468ff21813-openshift-service-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:58:15.347747 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.347755 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6d338bd-f164-45a8-ba86-ef468ff21813-maas-api-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:58:15.622936 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.622873 2572 generic.go:356] "Generic (PLEG): container finished" podID="c6d338bd-f164-45a8-ba86-ef468ff21813" containerID="4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7" exitCode=0 Jul 06 19:58:15.622936 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.622938 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" event={"ID":"c6d338bd-f164-45a8-ba86-ef468ff21813","Type":"ContainerDied","Data":"4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7"} Jul 06 19:58:15.623159 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.622954 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" Jul 06 19:58:15.623159 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.622992 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb" event={"ID":"c6d338bd-f164-45a8-ba86-ef468ff21813","Type":"ContainerDied","Data":"4884762e2e21fb4653948dac881d4686e3b7b23c708e939d831a5e3ca6bcd342"} Jul 06 19:58:15.623159 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.623022 2572 scope.go:117] "RemoveContainer" containerID="4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7" Jul 06 19:58:15.632486 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.632463 2572 scope.go:117] "RemoveContainer" containerID="4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7" Jul 06 19:58:15.632773 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:58:15.632754 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7\": container with ID starting with 4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7 not found: ID does not exist" containerID="4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7" Jul 06 19:58:15.632839 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.632783 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7"} err="failed to get container status \"4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7\": rpc error: code = NotFound desc = could not find container \"4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7\": container with ID starting with 4100393c4e0fb0f6c41fc8da70d4d86d1876c0c9afdf9f2f231f9ffb121274f7 not found: ID does not exist" Jul 06 19:58:15.646550 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.646516 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb"] Jul 06 19:58:15.650612 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:15.650585 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-a04b5bb9-7ffdcc48bf-bdvkb"] Jul 06 19:58:16.175214 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:16.175183 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6d338bd-f164-45a8-ba86-ef468ff21813" path="/var/lib/kubelet/pods/c6d338bd-f164-45a8-ba86-ef468ff21813/volumes" Jul 06 19:58:17.909829 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:17.909796 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:18.008750 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:18.008714 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:21.182393 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:21.182358 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:21.256177 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:21.256139 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:21.462533 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:21.462495 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:21.686128 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:21.686070 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:21.818161 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:21.818068 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:21.915319 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:21.915284 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:22.024922 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:22.024868 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:28.164556 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:28.164516 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:28.283835 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:28.283797 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:28.370900 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:28.370860 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:28.534377 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:28.534342 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:34.441547 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:34.441512 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:34.553299 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:34.553266 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:39.627222 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:39.627174 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:41.332505 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:41.332465 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:41.809526 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:41.809491 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:41.914039 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:41.914002 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:44.597092 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.597053 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:44.693058 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.693025 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:44.697310 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.697287 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr"] Jul 06 19:58:44.697643 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.697632 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d338bd-f164-45a8-ba86-ef468ff21813" containerName="maas-api" Jul 06 19:58:44.697689 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.697645 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d338bd-f164-45a8-ba86-ef468ff21813" containerName="maas-api" Jul 06 19:58:44.697728 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.697706 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d338bd-f164-45a8-ba86-ef468ff21813" containerName="maas-api" Jul 06 19:58:44.700834 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.700817 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:44.703196 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.703179 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-2b7eaa8b-serving-cert\"" Jul 06 19:58:44.712689 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.712662 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr"] Jul 06 19:58:44.796606 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.796560 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:44.815921 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.815880 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls\") pod \"maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:44.816108 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.815966 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5x2q\" (UniqueName: \"kubernetes.io/projected/4e46dd70-dab4-4859-af7a-7a174cb4393c-kube-api-access-m5x2q\") pod \"maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:44.816108 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.816056 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e46dd70-dab4-4859-af7a-7a174cb4393c-openshift-service-ca\") pod \"maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:44.916747 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.916646 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5x2q\" (UniqueName: \"kubernetes.io/projected/4e46dd70-dab4-4859-af7a-7a174cb4393c-kube-api-access-m5x2q\") pod \"maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:44.916747 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.916721 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e46dd70-dab4-4859-af7a-7a174cb4393c-openshift-service-ca\") pod \"maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:44.917030 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.916760 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls\") pod \"maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:44.917030 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:58:44.916895 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-2b7eaa8b-serving-cert: secret "maas-api-e2e-mt-2b7eaa8b-serving-cert" not found Jul 06 19:58:44.917030 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:58:44.917011 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls podName:4e46dd70-dab4-4859-af7a-7a174cb4393c nodeName:}" failed. No retries permitted until 2026-07-06 19:58:45.416990605 +0000 UTC m=+2035.922275621 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls") pod "maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" (UID: "4e46dd70-dab4-4859-af7a-7a174cb4393c") : secret "maas-api-e2e-mt-2b7eaa8b-serving-cert" not found Jul 06 19:58:44.917522 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.917500 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e46dd70-dab4-4859-af7a-7a174cb4393c-openshift-service-ca\") pod \"maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:44.928178 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:44.928149 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5x2q\" (UniqueName: \"kubernetes.io/projected/4e46dd70-dab4-4859-af7a-7a174cb4393c-kube-api-access-m5x2q\") pod \"maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:45.293259 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:45.293221 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:45.393423 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:45.393381 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:45.421632 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:45.421588 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls\") pod \"maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:45.421828 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:58:45.421739 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-2b7eaa8b-serving-cert: secret "maas-api-e2e-mt-2b7eaa8b-serving-cert" not found Jul 06 19:58:45.421828 ip-10-0-131-35 kubenswrapper[2572]: E0706 19:58:45.421819 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls podName:4e46dd70-dab4-4859-af7a-7a174cb4393c nodeName:}" failed. No retries permitted until 2026-07-06 19:58:46.421803011 +0000 UTC m=+2036.927088026 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls") pod "maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" (UID: "4e46dd70-dab4-4859-af7a-7a174cb4393c") : secret "maas-api-e2e-mt-2b7eaa8b-serving-cert" not found Jul 06 19:58:45.642776 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:45.642674 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:46.431227 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:46.431192 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls\") pod \"maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:46.433773 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:46.433748 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls\") pod \"maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:46.512869 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:46.512825 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:46.663767 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:46.663743 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr"] Jul 06 19:58:46.666256 ip-10-0-131-35 kubenswrapper[2572]: W0706 19:58:46.666224 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e46dd70_dab4_4859_af7a_7a174cb4393c.slice/crio-81be6a0a17e837084e7f2576e26d6703b54635662b0af4c8153a1bef4ce549b0 WatchSource:0}: Error finding container 81be6a0a17e837084e7f2576e26d6703b54635662b0af4c8153a1bef4ce549b0: Status 404 returned error can't find the container with id 81be6a0a17e837084e7f2576e26d6703b54635662b0af4c8153a1bef4ce549b0 Jul 06 19:58:46.785572 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:46.785536 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" event={"ID":"4e46dd70-dab4-4859-af7a-7a174cb4393c","Type":"ContainerStarted","Data":"81be6a0a17e837084e7f2576e26d6703b54635662b0af4c8153a1bef4ce549b0"} Jul 06 19:58:47.791562 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:47.791525 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" event={"ID":"4e46dd70-dab4-4859-af7a-7a174cb4393c","Type":"ContainerStarted","Data":"780a1adf36ba60d657be82f881bd0cffbe9c755bb3ef7ef299da425f611a5b43"} Jul 06 19:58:47.791993 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:47.791587 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:58:47.813710 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:47.813658 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" podStartSLOduration=3.387056666 podStartE2EDuration="3.813640249s" totalImagesPullingTime="426.583583ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:58:47.812081778 +0000 UTC m=+2038.317366834" watchObservedRunningTime="2026-07-06 19:58:47.813640249 +0000 UTC m=+2038.318925289" Jul 06 19:58:49.114078 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:49.114035 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:52.939212 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:52.939176 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:58:53.801478 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:58:53.801452 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:59:16.474227 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:16.474133 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:16.770754 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:16.770666 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr"] Jul 06 19:59:16.770976 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:16.770898 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" podUID="4e46dd70-dab4-4859-af7a-7a174cb4393c" containerName="maas-api" containerID="cri-o://780a1adf36ba60d657be82f881bd0cffbe9c755bb3ef7ef299da425f611a5b43" gracePeriod=30 Jul 06 19:59:16.801376 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:16.801341 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:16.937808 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:16.937769 2572 generic.go:356] "Generic (PLEG): container finished" podID="4e46dd70-dab4-4859-af7a-7a174cb4393c" containerID="780a1adf36ba60d657be82f881bd0cffbe9c755bb3ef7ef299da425f611a5b43" exitCode=0 Jul 06 19:59:16.938067 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:16.937837 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" event={"ID":"4e46dd70-dab4-4859-af7a-7a174cb4393c","Type":"ContainerDied","Data":"780a1adf36ba60d657be82f881bd0cffbe9c755bb3ef7ef299da425f611a5b43"} Jul 06 19:59:17.021228 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.021170 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:59:17.110375 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.110320 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5x2q\" (UniqueName: \"kubernetes.io/projected/4e46dd70-dab4-4859-af7a-7a174cb4393c-kube-api-access-m5x2q\") pod \"4e46dd70-dab4-4859-af7a-7a174cb4393c\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " Jul 06 19:59:17.110585 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.110391 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e46dd70-dab4-4859-af7a-7a174cb4393c-openshift-service-ca\") pod \"4e46dd70-dab4-4859-af7a-7a174cb4393c\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " Jul 06 19:59:17.110585 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.110479 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls\") pod \"4e46dd70-dab4-4859-af7a-7a174cb4393c\" (UID: \"4e46dd70-dab4-4859-af7a-7a174cb4393c\") " Jul 06 19:59:17.110766 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.110739 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e46dd70-dab4-4859-af7a-7a174cb4393c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4e46dd70-dab4-4859-af7a-7a174cb4393c" (UID: "4e46dd70-dab4-4859-af7a-7a174cb4393c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:59:17.112600 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.112563 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4e46dd70-dab4-4859-af7a-7a174cb4393c" (UID: "4e46dd70-dab4-4859-af7a-7a174cb4393c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:59:17.112710 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.112634 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e46dd70-dab4-4859-af7a-7a174cb4393c-kube-api-access-m5x2q" (OuterVolumeSpecName: "kube-api-access-m5x2q") pod "4e46dd70-dab4-4859-af7a-7a174cb4393c" (UID: "4e46dd70-dab4-4859-af7a-7a174cb4393c"). InnerVolumeSpecName "kube-api-access-m5x2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:59:17.211964 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.211890 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e46dd70-dab4-4859-af7a-7a174cb4393c-maas-api-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:59:17.211964 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.211958 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5x2q\" (UniqueName: \"kubernetes.io/projected/4e46dd70-dab4-4859-af7a-7a174cb4393c-kube-api-access-m5x2q\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:59:17.211964 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.211969 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e46dd70-dab4-4859-af7a-7a174cb4393c-openshift-service-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 19:59:17.945248 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.945217 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" Jul 06 19:59:17.945694 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.945253 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr" event={"ID":"4e46dd70-dab4-4859-af7a-7a174cb4393c","Type":"ContainerDied","Data":"81be6a0a17e837084e7f2576e26d6703b54635662b0af4c8153a1bef4ce549b0"} Jul 06 19:59:17.945694 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.945315 2572 scope.go:117] "RemoveContainer" containerID="780a1adf36ba60d657be82f881bd0cffbe9c755bb3ef7ef299da425f611a5b43" Jul 06 19:59:17.968741 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.968702 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr"] Jul 06 19:59:17.973479 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:17.973455 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-2b7eaa8b-79c8fd7cfd-vqprr"] Jul 06 19:59:18.176290 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:18.176260 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e46dd70-dab4-4859-af7a-7a174cb4393c" path="/var/lib/kubelet/pods/4e46dd70-dab4-4859-af7a-7a174cb4393c/volumes" Jul 06 19:59:22.527402 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:22.527361 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:27.605200 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:27.605158 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:30.523254 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:30.523220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:30.613448 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:30.613415 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 19:59:50.084697 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:50.084668 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:59:50.089259 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:50.089228 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 19:59:50.089497 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:50.089480 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 19:59:50.094024 ip-10-0-131-35 kubenswrapper[2572]: I0706 19:59:50.094004 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 20:00:04.712060 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:04.712021 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:08.164458 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:08.164425 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:11.025753 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:11.025710 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:34.278544 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:34.278511 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:34.376395 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:34.376355 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:34.554847 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:34.554757 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:34.785297 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:34.785261 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:35.847675 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:35.847637 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:35.945577 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:35.945536 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:36.036322 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:36.036287 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:36.783533 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:36.783488 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:36.887964 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:36.887928 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:36.985623 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:36.985579 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:38.380850 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:38.380811 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:44.512464 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:44.512378 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:44.610956 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:44.610921 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:45.199521 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:45.199480 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:52.735120 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:52.735084 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:00:53.391300 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:00:53.391260 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:12.496719 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:12.496682 2572 generic.go:356] "Generic (PLEG): container finished" podID="bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa" containerID="6e2423185bad3453203495cf23b80680b604aacc070d5c6bab8f0eefeb132ee7" exitCode=0 Jul 06 20:01:12.497218 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:12.496736 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" event={"ID":"bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa","Type":"ContainerDied","Data":"6e2423185bad3453203495cf23b80680b604aacc070d5c6bab8f0eefeb132ee7"} Jul 06 20:01:12.497218 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:12.497111 2572 scope.go:117] "RemoveContainer" containerID="6e2423185bad3453203495cf23b80680b604aacc070d5c6bab8f0eefeb132ee7" Jul 06 20:01:13.503059 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.503021 2572 generic.go:356] "Generic (PLEG): container finished" podID="8f8ff396-377b-4cd6-a4f2-9b26caeab604" containerID="250d54c59310e84a65fe3b58a9417f8384117606c47f71374131a5192f71fa44" exitCode=1 Jul 06 20:01:13.503506 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.503099 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" event={"ID":"8f8ff396-377b-4cd6-a4f2-9b26caeab604","Type":"ContainerDied","Data":"250d54c59310e84a65fe3b58a9417f8384117606c47f71374131a5192f71fa44"} Jul 06 20:01:13.503506 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.503478 2572 scope.go:117] "RemoveContainer" containerID="250d54c59310e84a65fe3b58a9417f8384117606c47f71374131a5192f71fa44" Jul 06 20:01:13.503861 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.503844 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:01:13.504820 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.504795 2572 generic.go:356] "Generic (PLEG): container finished" podID="fad8e3b0-82a9-4d95-8807-aa3819bcb9e7" containerID="36390896e13fd7bda86fad6dbaf04b34f3b55b5f193221cb0da329ebf1fd17a4" exitCode=1 Jul 06 20:01:13.504961 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.504870 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" event={"ID":"fad8e3b0-82a9-4d95-8807-aa3819bcb9e7","Type":"ContainerDied","Data":"36390896e13fd7bda86fad6dbaf04b34f3b55b5f193221cb0da329ebf1fd17a4"} Jul 06 20:01:13.505304 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.505284 2572 scope.go:117] "RemoveContainer" containerID="36390896e13fd7bda86fad6dbaf04b34f3b55b5f193221cb0da329ebf1fd17a4" Jul 06 20:01:13.506678 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.506655 2572 generic.go:356] "Generic (PLEG): container finished" podID="233283df-b179-4063-97ec-96608510f7d7" containerID="8597863ef737f47585763a2abeeb6d167c4cb6c9805d3ca60a4e9a0281ebac82" exitCode=1 Jul 06 20:01:13.506757 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.506723 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" event={"ID":"233283df-b179-4063-97ec-96608510f7d7","Type":"ContainerDied","Data":"8597863ef737f47585763a2abeeb6d167c4cb6c9805d3ca60a4e9a0281ebac82"} Jul 06 20:01:13.507134 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.507115 2572 scope.go:117] "RemoveContainer" containerID="8597863ef737f47585763a2abeeb6d167c4cb6c9805d3ca60a4e9a0281ebac82" Jul 06 20:01:13.508995 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.508957 2572 generic.go:356] "Generic (PLEG): container finished" podID="8bf46798-5ef5-4e66-b54a-5928af921668" containerID="4fd91659dccbc1345f987f9346058b6888007f157b3b886590c0b9f2530c6446" exitCode=1 Jul 06 20:01:13.509102 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.509043 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" event={"ID":"8bf46798-5ef5-4e66-b54a-5928af921668","Type":"ContainerDied","Data":"4fd91659dccbc1345f987f9346058b6888007f157b3b886590c0b9f2530c6446"} Jul 06 20:01:13.509439 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.509423 2572 scope.go:117] "RemoveContainer" containerID="4fd91659dccbc1345f987f9346058b6888007f157b3b886590c0b9f2530c6446" Jul 06 20:01:13.511137 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.511111 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 20:01:13.511300 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.511279 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" event={"ID":"bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa","Type":"ContainerStarted","Data":"798476ad668e5efd0e2c1464987dd4f2dd274c72429a30c0b211127837832966"} Jul 06 20:01:13.511495 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:13.511483 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 20:01:14.518455 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:14.518419 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" event={"ID":"fad8e3b0-82a9-4d95-8807-aa3819bcb9e7","Type":"ContainerStarted","Data":"f399d5bc8ad4584cd8d2fe51d98eca223064916bec05f54abd8967b873142090"} Jul 06 20:01:14.518964 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:14.518638 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 20:01:14.520331 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:14.520305 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" event={"ID":"233283df-b179-4063-97ec-96608510f7d7","Type":"ContainerStarted","Data":"440e8af37ba5ce7c75d3845e3eafe26fc64a5429ac9775d08a818cdf94e1779f"} Jul 06 20:01:14.520509 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:14.520489 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" Jul 06 20:01:14.522611 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:14.522590 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" event={"ID":"8bf46798-5ef5-4e66-b54a-5928af921668","Type":"ContainerStarted","Data":"e0a495f9908925eb090f12290c006319542da5cc06e79496fc7f611438104fa5"} Jul 06 20:01:14.522821 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:14.522799 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 20:01:14.524545 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:14.524518 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" event={"ID":"8f8ff396-377b-4cd6-a4f2-9b26caeab604","Type":"ContainerStarted","Data":"63ec0ecd1841867ad7c35b72019fdb9870d863e1a662df0a10131c919c08e9a7"} Jul 06 20:01:14.524750 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:14.524737 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 20:01:16.508191 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:16.508153 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:16.623212 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:16.623171 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:16.755291 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:16.755255 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:17.465266 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:17.465228 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:24.526777 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:24.526742 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-49k9p" Jul 06 20:01:25.532289 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:25.532255 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jm6m" Jul 06 20:01:25.532660 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:25.532345 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-849ccdb5-xh49b" Jul 06 20:01:34.764578 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:34.764540 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:34.862576 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:34.862515 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:37.574161 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:37.574129 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:01:45.531942 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:45.531884 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x2577" Jul 06 20:01:45.535168 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:45.535148 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lwftm" Jul 06 20:01:58.607977 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:01:58.607938 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:05.025202 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:05.025161 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:05.123601 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:05.123570 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:05.387197 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:05.387106 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:05.747302 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:05.747229 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:06.478202 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:06.478169 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:06.633508 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:06.633471 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:08.577824 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.577784 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:08.690694 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.690649 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:08.720708 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.720674 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv"] Jul 06 20:02:08.721069 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.721056 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e46dd70-dab4-4859-af7a-7a174cb4393c" containerName="maas-api" Jul 06 20:02:08.721120 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.721070 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e46dd70-dab4-4859-af7a-7a174cb4393c" containerName="maas-api" Jul 06 20:02:08.721156 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.721131 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e46dd70-dab4-4859-af7a-7a174cb4393c" containerName="maas-api" Jul 06 20:02:08.724555 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.724537 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:08.727144 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.727122 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-749ce0-serving-cert\"" Jul 06 20:02:08.744493 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.744452 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv"] Jul 06 20:02:08.781615 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.781582 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d8df13f-fa70-45ef-80c0-398d9b775607-maas-api-tls\") pod \"maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:08.781822 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.781649 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8df13f-fa70-45ef-80c0-398d9b775607-openshift-service-ca\") pod \"maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:08.781822 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.781701 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgsw2\" (UniqueName: \"kubernetes.io/projected/0d8df13f-fa70-45ef-80c0-398d9b775607-kube-api-access-zgsw2\") pod \"maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:08.821037 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.821000 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:08.882818 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.882720 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8df13f-fa70-45ef-80c0-398d9b775607-openshift-service-ca\") pod \"maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:08.882818 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.882769 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgsw2\" (UniqueName: \"kubernetes.io/projected/0d8df13f-fa70-45ef-80c0-398d9b775607-kube-api-access-zgsw2\") pod \"maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:08.883069 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.882850 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d8df13f-fa70-45ef-80c0-398d9b775607-maas-api-tls\") pod \"maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:08.883069 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:02:08.883031 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-749ce0-serving-cert: secret "maas-api-e2e-inf-a-749ce0-serving-cert" not found Jul 06 20:02:08.883142 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:02:08.883102 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d8df13f-fa70-45ef-80c0-398d9b775607-maas-api-tls podName:0d8df13f-fa70-45ef-80c0-398d9b775607 nodeName:}" failed. No retries permitted until 2026-07-06 20:02:09.383086836 +0000 UTC m=+2239.888371852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0d8df13f-fa70-45ef-80c0-398d9b775607-maas-api-tls") pod "maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" (UID: "0d8df13f-fa70-45ef-80c0-398d9b775607") : secret "maas-api-e2e-inf-a-749ce0-serving-cert" not found Jul 06 20:02:08.883506 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.883488 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8df13f-fa70-45ef-80c0-398d9b775607-openshift-service-ca\") pod \"maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:08.892992 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:08.892966 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgsw2\" (UniqueName: \"kubernetes.io/projected/0d8df13f-fa70-45ef-80c0-398d9b775607-kube-api-access-zgsw2\") pod \"maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:09.235107 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:09.235073 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:09.387964 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:09.387888 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d8df13f-fa70-45ef-80c0-398d9b775607-maas-api-tls\") pod \"maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:09.390419 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:09.390392 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d8df13f-fa70-45ef-80c0-398d9b775607-maas-api-tls\") pod \"maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:09.529708 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:09.529612 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:09.635160 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:09.635119 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:09.780324 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:09.780226 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:09.803528 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:09.803503 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv"] Jul 06 20:02:09.805774 ip-10-0-131-35 kubenswrapper[2572]: W0706 20:02:09.805745 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d8df13f_fa70_45ef_80c0_398d9b775607.slice/crio-54728de232c4ffecee87e5f8d16df6ed4790d4b635550effb78e9f3a1f75363b WatchSource:0}: Error finding container 54728de232c4ffecee87e5f8d16df6ed4790d4b635550effb78e9f3a1f75363b: Status 404 returned error can't find the container with id 54728de232c4ffecee87e5f8d16df6ed4790d4b635550effb78e9f3a1f75363b Jul 06 20:02:09.823640 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:09.823608 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" event={"ID":"0d8df13f-fa70-45ef-80c0-398d9b775607","Type":"ContainerStarted","Data":"54728de232c4ffecee87e5f8d16df6ed4790d4b635550effb78e9f3a1f75363b"} Jul 06 20:02:10.830426 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:10.830387 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" event={"ID":"0d8df13f-fa70-45ef-80c0-398d9b775607","Type":"ContainerStarted","Data":"168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340"} Jul 06 20:02:10.830880 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:10.830522 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:10.853222 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:10.853158 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" podStartSLOduration=2.327896004 podStartE2EDuration="2.853139115s" totalImagesPullingTime="525.243111ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:02:10.851205216 +0000 UTC m=+2241.356490255" watchObservedRunningTime="2026-07-06 20:02:10.853139115 +0000 UTC m=+2241.358424153" Jul 06 20:02:13.058949 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:13.058850 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:13.393923 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:13.393797 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:13.730591 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:13.730553 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:14.110718 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:14.110600 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:16.561366 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.561330 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:16.637004 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.636925 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk"] Jul 06 20:02:16.642404 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.642371 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk"] Jul 06 20:02:16.642593 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.642508 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:16.645220 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.645193 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-fc607d-serving-cert\"" Jul 06 20:02:16.689162 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.689128 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:16.759019 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.758979 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/769f893a-f142-430f-931b-bb949c37840a-maas-api-tls\") pod \"maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:16.759209 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.759127 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/769f893a-f142-430f-931b-bb949c37840a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:16.759209 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.759174 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqjl6\" (UniqueName: \"kubernetes.io/projected/769f893a-f142-430f-931b-bb949c37840a-kube-api-access-xqjl6\") pod \"maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:16.803192 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.803161 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:16.842893 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.842801 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:02:16.860456 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.860422 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/769f893a-f142-430f-931b-bb949c37840a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:16.860456 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.860456 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqjl6\" (UniqueName: \"kubernetes.io/projected/769f893a-f142-430f-931b-bb949c37840a-kube-api-access-xqjl6\") pod \"maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:16.860718 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.860514 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/769f893a-f142-430f-931b-bb949c37840a-maas-api-tls\") pod \"maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:16.860718 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:02:16.860620 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-fc607d-serving-cert: secret "maas-api-e2e-inf-b-fc607d-serving-cert" not found Jul 06 20:02:16.860718 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:02:16.860694 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/769f893a-f142-430f-931b-bb949c37840a-maas-api-tls podName:769f893a-f142-430f-931b-bb949c37840a nodeName:}" failed. No retries permitted until 2026-07-06 20:02:17.360673208 +0000 UTC m=+2247.865958224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/769f893a-f142-430f-931b-bb949c37840a-maas-api-tls") pod "maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" (UID: "769f893a-f142-430f-931b-bb949c37840a") : secret "maas-api-e2e-inf-b-fc607d-serving-cert" not found Jul 06 20:02:16.861150 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.861125 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/769f893a-f142-430f-931b-bb949c37840a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:16.888876 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:16.888844 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqjl6\" (UniqueName: \"kubernetes.io/projected/769f893a-f142-430f-931b-bb949c37840a-kube-api-access-xqjl6\") pod \"maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:17.239060 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:17.239020 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:17.366365 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:17.366326 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/769f893a-f142-430f-931b-bb949c37840a-maas-api-tls\") pod \"maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:17.368967 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:17.368938 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/769f893a-f142-430f-931b-bb949c37840a-maas-api-tls\") pod \"maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:17.375385 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:17.375341 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:17.495224 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:17.495109 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:17.556042 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:17.556006 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:17.710429 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:17.710369 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk"] Jul 06 20:02:17.712840 ip-10-0-131-35 kubenswrapper[2572]: W0706 20:02:17.712793 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod769f893a_f142_430f_931b_bb949c37840a.slice/crio-bd3e34770746d8ac17664806c46f86cc30fd5b10442722d91eece7d5c6c7e2b7 WatchSource:0}: Error finding container bd3e34770746d8ac17664806c46f86cc30fd5b10442722d91eece7d5c6c7e2b7: Status 404 returned error can't find the container with id bd3e34770746d8ac17664806c46f86cc30fd5b10442722d91eece7d5c6c7e2b7 Jul 06 20:02:17.867659 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:17.867573 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" event={"ID":"769f893a-f142-430f-931b-bb949c37840a","Type":"ContainerStarted","Data":"bd3e34770746d8ac17664806c46f86cc30fd5b10442722d91eece7d5c6c7e2b7"} Jul 06 20:02:17.939680 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:17.939641 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:18.113679 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:18.113646 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:18.875204 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:18.875171 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" event={"ID":"769f893a-f142-430f-931b-bb949c37840a","Type":"ContainerStarted","Data":"cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11"} Jul 06 20:02:18.875646 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:18.875235 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:18.897369 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:18.897317 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" podStartSLOduration=2.564388371 podStartE2EDuration="2.897302048s" totalImagesPullingTime="332.913677ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:02:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:02:18.896067285 +0000 UTC m=+2249.401352328" watchObservedRunningTime="2026-07-06 20:02:18.897302048 +0000 UTC m=+2249.402587085" Jul 06 20:02:21.180771 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:21.180736 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:24.885784 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:24.885756 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:02:30.241230 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:30.241187 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:30.339575 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:30.339533 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:35.745569 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:35.745531 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:38.211443 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:38.211403 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:02:38.312153 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:02:38.312108 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:03:15.711874 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:03:15.711832 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:03:49.087566 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:03:49.087483 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:14.782325 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:14.782271 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:15.978180 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:15.978123 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:16.131111 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:16.131075 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:16.531376 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:16.531340 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:16.657002 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:16.656964 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:16.757090 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:16.757051 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:16.827009 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:16.826931 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv"] Jul 06 20:04:16.827161 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:16.827139 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" podUID="0d8df13f-fa70-45ef-80c0-398d9b775607" containerName="maas-api" containerID="cri-o://168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340" gracePeriod=30 Jul 06 20:04:16.837716 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:16.837677 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" podUID="0d8df13f-fa70-45ef-80c0-398d9b775607" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.50:8443/health\": dial tcp 10.132.0.50:8443: connect: connection refused" Jul 06 20:04:16.859092 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:16.859062 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:17.080405 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.080331 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:04:17.232636 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.232602 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d8df13f-fa70-45ef-80c0-398d9b775607-maas-api-tls\") pod \"0d8df13f-fa70-45ef-80c0-398d9b775607\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " Jul 06 20:04:17.232795 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.232650 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8df13f-fa70-45ef-80c0-398d9b775607-openshift-service-ca\") pod \"0d8df13f-fa70-45ef-80c0-398d9b775607\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " Jul 06 20:04:17.232795 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.232679 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgsw2\" (UniqueName: \"kubernetes.io/projected/0d8df13f-fa70-45ef-80c0-398d9b775607-kube-api-access-zgsw2\") pod \"0d8df13f-fa70-45ef-80c0-398d9b775607\" (UID: \"0d8df13f-fa70-45ef-80c0-398d9b775607\") " Jul 06 20:04:17.233123 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.233093 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d8df13f-fa70-45ef-80c0-398d9b775607-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0d8df13f-fa70-45ef-80c0-398d9b775607" (UID: "0d8df13f-fa70-45ef-80c0-398d9b775607"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:04:17.234950 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.234872 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d8df13f-fa70-45ef-80c0-398d9b775607-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0d8df13f-fa70-45ef-80c0-398d9b775607" (UID: "0d8df13f-fa70-45ef-80c0-398d9b775607"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:04:17.238039 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.238006 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d8df13f-fa70-45ef-80c0-398d9b775607-kube-api-access-zgsw2" (OuterVolumeSpecName: "kube-api-access-zgsw2") pod "0d8df13f-fa70-45ef-80c0-398d9b775607" (UID: "0d8df13f-fa70-45ef-80c0-398d9b775607"). InnerVolumeSpecName "kube-api-access-zgsw2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:04:17.333347 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.333265 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d8df13f-fa70-45ef-80c0-398d9b775607-maas-api-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:04:17.333347 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.333293 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d8df13f-fa70-45ef-80c0-398d9b775607-openshift-service-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:04:17.333347 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.333303 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgsw2\" (UniqueName: \"kubernetes.io/projected/0d8df13f-fa70-45ef-80c0-398d9b775607-kube-api-access-zgsw2\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:04:17.489756 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.489722 2572 generic.go:356] "Generic (PLEG): container finished" podID="0d8df13f-fa70-45ef-80c0-398d9b775607" containerID="168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340" exitCode=0 Jul 06 20:04:17.489974 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.489769 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" Jul 06 20:04:17.489974 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.489807 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" event={"ID":"0d8df13f-fa70-45ef-80c0-398d9b775607","Type":"ContainerDied","Data":"168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340"} Jul 06 20:04:17.489974 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.489874 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv" event={"ID":"0d8df13f-fa70-45ef-80c0-398d9b775607","Type":"ContainerDied","Data":"54728de232c4ffecee87e5f8d16df6ed4790d4b635550effb78e9f3a1f75363b"} Jul 06 20:04:17.489974 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.489918 2572 scope.go:117] "RemoveContainer" containerID="168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340" Jul 06 20:04:17.499486 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.499461 2572 scope.go:117] "RemoveContainer" containerID="168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340" Jul 06 20:04:17.499788 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:04:17.499770 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340\": container with ID starting with 168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340 not found: ID does not exist" containerID="168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340" Jul 06 20:04:17.499857 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.499800 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340"} err="failed to get container status \"168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340\": rpc error: code = NotFound desc = could not find container \"168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340\": container with ID starting with 168d27d7f8c53bd0e23996f4973e9c339b21f900c9bec064c315a58ca0598340 not found: ID does not exist" Jul 06 20:04:17.512775 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.512741 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv"] Jul 06 20:04:17.517556 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:17.517527 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-749ce0-64c4b85ccb-sjppv"] Jul 06 20:04:18.175632 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:18.175599 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d8df13f-fa70-45ef-80c0-398d9b775607" path="/var/lib/kubelet/pods/0d8df13f-fa70-45ef-80c0-398d9b775607/volumes" Jul 06 20:04:21.610284 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:21.610226 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:04:50.114730 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:50.114700 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 20:04:50.118922 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:50.118883 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 20:04:50.121036 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:50.121019 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 20:04:50.125109 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:50.125092 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 20:04:59.793086 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:04:59.793046 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:00.156224 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:00.156138 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:00.367385 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:00.367353 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:00.585405 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:00.585353 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:00.999295 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:00.999256 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:01.313765 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.313669 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:01.442937 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.442874 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:01.663590 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.663501 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk"] Jul 06 20:05:01.663764 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.663710 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" podUID="769f893a-f142-430f-931b-bb949c37840a" containerName="maas-api" containerID="cri-o://cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11" gracePeriod=30 Jul 06 20:05:01.709156 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.709110 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:01.921814 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.921755 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:05:01.944009 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.943960 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqjl6\" (UniqueName: \"kubernetes.io/projected/769f893a-f142-430f-931b-bb949c37840a-kube-api-access-xqjl6\") pod \"769f893a-f142-430f-931b-bb949c37840a\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " Jul 06 20:05:01.944199 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.944089 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/769f893a-f142-430f-931b-bb949c37840a-maas-api-tls\") pod \"769f893a-f142-430f-931b-bb949c37840a\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " Jul 06 20:05:01.944199 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.944127 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/769f893a-f142-430f-931b-bb949c37840a-openshift-service-ca\") pod \"769f893a-f142-430f-931b-bb949c37840a\" (UID: \"769f893a-f142-430f-931b-bb949c37840a\") " Jul 06 20:05:01.944559 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.944534 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/769f893a-f142-430f-931b-bb949c37840a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "769f893a-f142-430f-931b-bb949c37840a" (UID: "769f893a-f142-430f-931b-bb949c37840a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:05:01.946191 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.946165 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/769f893a-f142-430f-931b-bb949c37840a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "769f893a-f142-430f-931b-bb949c37840a" (UID: "769f893a-f142-430f-931b-bb949c37840a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:05:01.946308 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:01.946255 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/769f893a-f142-430f-931b-bb949c37840a-kube-api-access-xqjl6" (OuterVolumeSpecName: "kube-api-access-xqjl6") pod "769f893a-f142-430f-931b-bb949c37840a" (UID: "769f893a-f142-430f-931b-bb949c37840a"). InnerVolumeSpecName "kube-api-access-xqjl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:05:02.045626 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.045589 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqjl6\" (UniqueName: \"kubernetes.io/projected/769f893a-f142-430f-931b-bb949c37840a-kube-api-access-xqjl6\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:05:02.045626 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.045620 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/769f893a-f142-430f-931b-bb949c37840a-maas-api-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:05:02.045626 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.045632 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/769f893a-f142-430f-931b-bb949c37840a-openshift-service-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:05:02.727362 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.727325 2572 generic.go:356] "Generic (PLEG): container finished" podID="769f893a-f142-430f-931b-bb949c37840a" containerID="cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11" exitCode=0 Jul 06 20:05:02.727551 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.727370 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" Jul 06 20:05:02.727551 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.727411 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" event={"ID":"769f893a-f142-430f-931b-bb949c37840a","Type":"ContainerDied","Data":"cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11"} Jul 06 20:05:02.727551 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.727463 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk" event={"ID":"769f893a-f142-430f-931b-bb949c37840a","Type":"ContainerDied","Data":"bd3e34770746d8ac17664806c46f86cc30fd5b10442722d91eece7d5c6c7e2b7"} Jul 06 20:05:02.727551 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.727485 2572 scope.go:117] "RemoveContainer" containerID="cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11" Jul 06 20:05:02.739002 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.738972 2572 scope.go:117] "RemoveContainer" containerID="cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11" Jul 06 20:05:02.739489 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:05:02.739467 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11\": container with ID starting with cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11 not found: ID does not exist" containerID="cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11" Jul 06 20:05:02.739573 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.739498 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11"} err="failed to get container status \"cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11\": rpc error: code = NotFound desc = could not find container \"cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11\": container with ID starting with cfce78d9ffe1d11aab8b7cd8eaaa0d4afb2c0943cdd817081ec0d7f4cbd6fe11 not found: ID does not exist" Jul 06 20:05:02.767396 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.767366 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk"] Jul 06 20:05:02.773458 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:02.773431 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fc607d-9589ccb65-5n9dk"] Jul 06 20:05:04.175804 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:04.175770 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="769f893a-f142-430f-931b-bb949c37840a" path="/var/lib/kubelet/pods/769f893a-f142-430f-931b-bb949c37840a/volumes" Jul 06 20:05:08.157360 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:08.157327 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:09.651413 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:09.651373 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:12.634893 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.634791 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:12.757657 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.757612 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:12.814789 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.814752 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p"] Jul 06 20:05:12.815399 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.815380 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d8df13f-fa70-45ef-80c0-398d9b775607" containerName="maas-api" Jul 06 20:05:12.815399 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.815401 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d8df13f-fa70-45ef-80c0-398d9b775607" containerName="maas-api" Jul 06 20:05:12.815562 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.815416 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="769f893a-f142-430f-931b-bb949c37840a" containerName="maas-api" Jul 06 20:05:12.815562 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.815425 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="769f893a-f142-430f-931b-bb949c37840a" containerName="maas-api" Jul 06 20:05:12.815562 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.815549 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d8df13f-fa70-45ef-80c0-398d9b775607" containerName="maas-api" Jul 06 20:05:12.815673 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.815565 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="769f893a-f142-430f-931b-bb949c37840a" containerName="maas-api" Jul 06 20:05:12.820094 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.820071 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:12.823528 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.823509 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-dcc969-serving-cert\"" Jul 06 20:05:12.842860 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.842830 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p"] Jul 06 20:05:12.880117 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.880087 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:12.937900 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.937801 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-openshift-service-ca\") pod \"maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:12.937900 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.937844 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-maas-api-tls\") pod \"maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:12.938117 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:12.937931 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfxlx\" (UniqueName: \"kubernetes.io/projected/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-kube-api-access-pfxlx\") pod \"maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:13.039053 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.039014 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-openshift-service-ca\") pod \"maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:13.039053 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.039056 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-maas-api-tls\") pod \"maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:13.039315 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.039093 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfxlx\" (UniqueName: \"kubernetes.io/projected/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-kube-api-access-pfxlx\") pod \"maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:13.039315 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:05:13.039196 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-dcc969-serving-cert: secret "maas-api-e2e-api-a-dcc969-serving-cert" not found Jul 06 20:05:13.039315 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:05:13.039273 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-maas-api-tls podName:d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df nodeName:}" failed. No retries permitted until 2026-07-06 20:05:13.539252139 +0000 UTC m=+2424.044537160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-maas-api-tls") pod "maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" (UID: "d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df") : secret "maas-api-e2e-api-a-dcc969-serving-cert" not found Jul 06 20:05:13.039798 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.039776 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-openshift-service-ca\") pod \"maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:13.054195 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.054170 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfxlx\" (UniqueName: \"kubernetes.io/projected/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-kube-api-access-pfxlx\") pod \"maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:13.416308 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.416271 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:13.543233 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.543199 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-maas-api-tls\") pod \"maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:13.545880 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.545843 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-maas-api-tls\") pod \"maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:13.731083 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.731051 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:13.783992 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.783896 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:13.887307 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.887267 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p"] Jul 06 20:05:13.889748 ip-10-0-131-35 kubenswrapper[2572]: W0706 20:05:13.889720 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd48c3df2_2b4d_4b2f_bfa6_4a2fbc5463df.slice/crio-17d8579e814a6ebbed1405a123fdf2bcf44d23e665857688ae2792da2814f20d WatchSource:0}: Error finding container 17d8579e814a6ebbed1405a123fdf2bcf44d23e665857688ae2792da2814f20d: Status 404 returned error can't find the container with id 17d8579e814a6ebbed1405a123fdf2bcf44d23e665857688ae2792da2814f20d Jul 06 20:05:13.904961 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:13.904929 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:14.789440 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:14.789405 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" event={"ID":"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df","Type":"ContainerStarted","Data":"b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e"} Jul 06 20:05:14.789808 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:14.789470 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:14.789808 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:14.789496 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" event={"ID":"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df","Type":"ContainerStarted","Data":"17d8579e814a6ebbed1405a123fdf2bcf44d23e665857688ae2792da2814f20d"} Jul 06 20:05:14.812733 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:14.812680 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" podStartSLOduration=2.2733712710000002 podStartE2EDuration="2.812661307s" totalImagesPullingTime="539.290036ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:05:14.811092126 +0000 UTC m=+2425.316377164" watchObservedRunningTime="2026-07-06 20:05:14.812661307 +0000 UTC m=+2425.317946345" Jul 06 20:05:17.012464 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:17.012418 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:17.483062 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:17.483032 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:17.824848 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:17.824740 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:18.231960 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:18.231888 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:20.550290 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:20.550253 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:20.777765 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:20.777730 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd"] Jul 06 20:05:20.781316 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:20.781292 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:20.783706 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:20.783686 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-ba1024-serving-cert\"" Jul 06 20:05:20.801511 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:20.801442 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:20.802632 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:20.802608 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd"] Jul 06 20:05:20.807542 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:20.807498 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:20.909431 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:20.909395 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f16de4c-9925-467e-96a5-e464d7c2f1da-openshift-service-ca\") pod \"maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:20.909612 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:20.909462 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr5sn\" (UniqueName: \"kubernetes.io/projected/7f16de4c-9925-467e-96a5-e464d7c2f1da-kube-api-access-qr5sn\") pod \"maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:20.909612 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:20.909572 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f16de4c-9925-467e-96a5-e464d7c2f1da-maas-api-tls\") pod \"maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:20.915863 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:20.915831 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:21.010620 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.010585 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f16de4c-9925-467e-96a5-e464d7c2f1da-openshift-service-ca\") pod \"maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:21.010837 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.010642 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr5sn\" (UniqueName: \"kubernetes.io/projected/7f16de4c-9925-467e-96a5-e464d7c2f1da-kube-api-access-qr5sn\") pod \"maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:21.010837 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.010702 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f16de4c-9925-467e-96a5-e464d7c2f1da-maas-api-tls\") pod \"maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:21.011361 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.011339 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f16de4c-9925-467e-96a5-e464d7c2f1da-openshift-service-ca\") pod \"maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:21.013211 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.013194 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f16de4c-9925-467e-96a5-e464d7c2f1da-maas-api-tls\") pod \"maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:21.018271 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.018243 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:21.022306 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.022288 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr5sn\" (UniqueName: \"kubernetes.io/projected/7f16de4c-9925-467e-96a5-e464d7c2f1da-kube-api-access-qr5sn\") pod \"maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:21.091339 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.091227 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:21.247427 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.247402 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd"] Jul 06 20:05:21.249332 ip-10-0-131-35 kubenswrapper[2572]: W0706 20:05:21.249299 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f16de4c_9925_467e_96a5_e464d7c2f1da.slice/crio-d170b7b0d37f8a832e9c304286b9856549c353b2c13a42a91405b33b8bc5064a WatchSource:0}: Error finding container d170b7b0d37f8a832e9c304286b9856549c353b2c13a42a91405b33b8bc5064a: Status 404 returned error can't find the container with id d170b7b0d37f8a832e9c304286b9856549c353b2c13a42a91405b33b8bc5064a Jul 06 20:05:21.714919 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.714887 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:21.827694 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.827659 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" event={"ID":"7f16de4c-9925-467e-96a5-e464d7c2f1da","Type":"ContainerStarted","Data":"19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7"} Jul 06 20:05:21.827899 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.827710 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" event={"ID":"7f16de4c-9925-467e-96a5-e464d7c2f1da","Type":"ContainerStarted","Data":"d170b7b0d37f8a832e9c304286b9856549c353b2c13a42a91405b33b8bc5064a"} Jul 06 20:05:21.827899 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.827796 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:21.830120 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.830097 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:21.858929 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:21.858854 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" podStartSLOduration=1.381938426 podStartE2EDuration="1.858835144s" totalImagesPullingTime="476.896718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:05:21.84996858 +0000 UTC m=+2432.355253620" watchObservedRunningTime="2026-07-06 20:05:21.858835144 +0000 UTC m=+2432.364120182" Jul 06 20:05:24.977311 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:24.977271 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:27.838677 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:27.838640 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:33.072264 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:33.072224 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:33.552862 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:33.552826 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:33.696537 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:33.696496 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:33.840795 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:33.840685 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p"] Jul 06 20:05:33.841114 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:33.841086 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" podUID="d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df" containerName="maas-api" containerID="cri-o://b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e" gracePeriod=30 Jul 06 20:05:33.876438 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:33.876391 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:34.104316 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.104249 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:34.232419 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.232382 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfxlx\" (UniqueName: \"kubernetes.io/projected/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-kube-api-access-pfxlx\") pod \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " Jul 06 20:05:34.232604 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.232428 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-maas-api-tls\") pod \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " Jul 06 20:05:34.232604 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.232563 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-openshift-service-ca\") pod \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\" (UID: \"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df\") " Jul 06 20:05:34.232987 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.232959 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df" (UID: "d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:05:34.234973 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.234942 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df" (UID: "d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:05:34.235095 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.235056 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-kube-api-access-pfxlx" (OuterVolumeSpecName: "kube-api-access-pfxlx") pod "d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df" (UID: "d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df"). InnerVolumeSpecName "kube-api-access-pfxlx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:05:34.333620 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.333583 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-maas-api-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:05:34.333620 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.333616 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-openshift-service-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:05:34.333620 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.333627 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfxlx\" (UniqueName: \"kubernetes.io/projected/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df-kube-api-access-pfxlx\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:05:34.865324 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.865287 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:34.895854 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.895817 2572 generic.go:356] "Generic (PLEG): container finished" podID="d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df" containerID="b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e" exitCode=0 Jul 06 20:05:34.896036 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.895877 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" Jul 06 20:05:34.896036 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.895899 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" event={"ID":"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df","Type":"ContainerDied","Data":"b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e"} Jul 06 20:05:34.896036 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.895982 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p" event={"ID":"d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df","Type":"ContainerDied","Data":"17d8579e814a6ebbed1405a123fdf2bcf44d23e665857688ae2792da2814f20d"} Jul 06 20:05:34.896036 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.896006 2572 scope.go:117] "RemoveContainer" containerID="b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e" Jul 06 20:05:34.905228 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.905212 2572 scope.go:117] "RemoveContainer" containerID="b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e" Jul 06 20:05:34.905520 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:05:34.905496 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e\": container with ID starting with b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e not found: ID does not exist" containerID="b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e" Jul 06 20:05:34.905619 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.905527 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e"} err="failed to get container status \"b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e\": rpc error: code = NotFound desc = could not find container \"b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e\": container with ID starting with b3f58c8863ad8b2441e495923e8d353946bb522abbc8de0f05ed306520983a1e not found: ID does not exist" Jul 06 20:05:34.920345 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.920315 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p"] Jul 06 20:05:34.930465 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:34.930436 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-dcc969-688cc7c57c-jdv2p"] Jul 06 20:05:35.317363 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:35.317315 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:36.175979 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:36.175950 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df" path="/var/lib/kubelet/pods/d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df/volumes" Jul 06 20:05:40.416093 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:40.416053 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:40.512181 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:40.512142 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:40.611692 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:40.611659 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:40.816696 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:40.816657 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:41.090311 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:41.090220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:41.441437 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:41.441348 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:41.600924 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:41.600828 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:41.696703 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:41.696611 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:41.899241 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:41.899204 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd"] Jul 06 20:05:41.899446 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:41.899424 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" podUID="7f16de4c-9925-467e-96a5-e464d7c2f1da" containerName="maas-api" containerID="cri-o://19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7" gracePeriod=30 Jul 06 20:05:41.942825 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:41.942787 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:42.165847 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.165815 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:42.306608 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.306503 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f16de4c-9925-467e-96a5-e464d7c2f1da-maas-api-tls\") pod \"7f16de4c-9925-467e-96a5-e464d7c2f1da\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " Jul 06 20:05:42.306608 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.306603 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f16de4c-9925-467e-96a5-e464d7c2f1da-openshift-service-ca\") pod \"7f16de4c-9925-467e-96a5-e464d7c2f1da\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " Jul 06 20:05:42.306823 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.306717 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr5sn\" (UniqueName: \"kubernetes.io/projected/7f16de4c-9925-467e-96a5-e464d7c2f1da-kube-api-access-qr5sn\") pod \"7f16de4c-9925-467e-96a5-e464d7c2f1da\" (UID: \"7f16de4c-9925-467e-96a5-e464d7c2f1da\") " Jul 06 20:05:42.306958 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.306898 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f16de4c-9925-467e-96a5-e464d7c2f1da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f16de4c-9925-467e-96a5-e464d7c2f1da" (UID: "7f16de4c-9925-467e-96a5-e464d7c2f1da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:05:42.307086 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.307071 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f16de4c-9925-467e-96a5-e464d7c2f1da-openshift-service-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:05:42.308812 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.308784 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f16de4c-9925-467e-96a5-e464d7c2f1da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f16de4c-9925-467e-96a5-e464d7c2f1da" (UID: "7f16de4c-9925-467e-96a5-e464d7c2f1da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:05:42.308949 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.308848 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f16de4c-9925-467e-96a5-e464d7c2f1da-kube-api-access-qr5sn" (OuterVolumeSpecName: "kube-api-access-qr5sn") pod "7f16de4c-9925-467e-96a5-e464d7c2f1da" (UID: "7f16de4c-9925-467e-96a5-e464d7c2f1da"). InnerVolumeSpecName "kube-api-access-qr5sn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:05:42.408202 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.408154 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f16de4c-9925-467e-96a5-e464d7c2f1da-maas-api-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:05:42.408202 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.408195 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr5sn\" (UniqueName: \"kubernetes.io/projected/7f16de4c-9925-467e-96a5-e464d7c2f1da-kube-api-access-qr5sn\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:05:42.939820 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.939783 2572 generic.go:356] "Generic (PLEG): container finished" podID="7f16de4c-9925-467e-96a5-e464d7c2f1da" containerID="19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7" exitCode=0 Jul 06 20:05:42.939820 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.939822 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" Jul 06 20:05:42.940373 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.939863 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" event={"ID":"7f16de4c-9925-467e-96a5-e464d7c2f1da","Type":"ContainerDied","Data":"19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7"} Jul 06 20:05:42.940373 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.939938 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd" event={"ID":"7f16de4c-9925-467e-96a5-e464d7c2f1da","Type":"ContainerDied","Data":"d170b7b0d37f8a832e9c304286b9856549c353b2c13a42a91405b33b8bc5064a"} Jul 06 20:05:42.940373 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.939965 2572 scope.go:117] "RemoveContainer" containerID="19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7" Jul 06 20:05:42.949496 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.949478 2572 scope.go:117] "RemoveContainer" containerID="19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7" Jul 06 20:05:42.949789 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:05:42.949770 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7\": container with ID starting with 19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7 not found: ID does not exist" containerID="19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7" Jul 06 20:05:42.949835 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.949800 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7"} err="failed to get container status \"19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7\": rpc error: code = NotFound desc = could not find container \"19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7\": container with ID starting with 19e3c3b7d94c2c57f8ff8323e10d1e60da65961669ff2d6164cbc56ca2bcc7b7 not found: ID does not exist" Jul 06 20:05:42.967348 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.967318 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd"] Jul 06 20:05:42.976374 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:42.976344 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-ba1024-77fc87cfc8-xq8kd"] Jul 06 20:05:44.176970 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:44.176936 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f16de4c-9925-467e-96a5-e464d7c2f1da" path="/var/lib/kubelet/pods/7f16de4c-9925-467e-96a5-e464d7c2f1da/volumes" Jul 06 20:05:48.414185 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:48.414156 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:48.537002 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:48.536960 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:48.726676 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:48.726640 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:48.986392 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:48.986302 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:49.339038 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:49.338948 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:49.741324 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:49.741286 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:50.112296 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:50.112204 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:52.748430 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.748385 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:52.843253 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.843200 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8"] Jul 06 20:05:52.843854 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.843828 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f16de4c-9925-467e-96a5-e464d7c2f1da" containerName="maas-api" Jul 06 20:05:52.843854 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.843856 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f16de4c-9925-467e-96a5-e464d7c2f1da" containerName="maas-api" Jul 06 20:05:52.844037 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.843923 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df" containerName="maas-api" Jul 06 20:05:52.844037 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.843934 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df" containerName="maas-api" Jul 06 20:05:52.844128 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.844046 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d48c3df2-2b4d-4b2f-bfa6-4a2fbc5463df" containerName="maas-api" Jul 06 20:05:52.844128 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.844059 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f16de4c-9925-467e-96a5-e464d7c2f1da" containerName="maas-api" Jul 06 20:05:52.848995 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.848971 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:52.851383 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.851361 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-7ba461-serving-cert\"" Jul 06 20:05:52.867367 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.867328 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8"] Jul 06 20:05:52.906681 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.906639 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:52.908710 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.908671 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41159632-e878-48ec-958d-7df6faa3ac60-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:52.908853 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.908798 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9pmx\" (UniqueName: \"kubernetes.io/projected/41159632-e878-48ec-958d-7df6faa3ac60-kube-api-access-l9pmx\") pod \"maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:52.908853 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:52.908846 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41159632-e878-48ec-958d-7df6faa3ac60-maas-api-tls\") pod \"maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:53.009872 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.009768 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9pmx\" (UniqueName: \"kubernetes.io/projected/41159632-e878-48ec-958d-7df6faa3ac60-kube-api-access-l9pmx\") pod \"maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:53.009872 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.009809 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41159632-e878-48ec-958d-7df6faa3ac60-maas-api-tls\") pod \"maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:53.009872 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.009833 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41159632-e878-48ec-958d-7df6faa3ac60-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:53.010547 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.010528 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41159632-e878-48ec-958d-7df6faa3ac60-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:53.012267 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.012246 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41159632-e878-48ec-958d-7df6faa3ac60-maas-api-tls\") pod \"maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:53.023410 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.023374 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9pmx\" (UniqueName: \"kubernetes.io/projected/41159632-e878-48ec-958d-7df6faa3ac60-kube-api-access-l9pmx\") pod \"maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:53.030554 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.030505 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:53.139317 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.139281 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:53.160216 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.160177 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:53.323035 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.323005 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8"] Jul 06 20:05:53.325706 ip-10-0-131-35 kubenswrapper[2572]: W0706 20:05:53.325674 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41159632_e878_48ec_958d_7df6faa3ac60.slice/crio-22582b0ae20ecfb5e18eb384b1b8eb548be99e3a477e9deb64c58b79101da682 WatchSource:0}: Error finding container 22582b0ae20ecfb5e18eb384b1b8eb548be99e3a477e9deb64c58b79101da682: Status 404 returned error can't find the container with id 22582b0ae20ecfb5e18eb384b1b8eb548be99e3a477e9deb64c58b79101da682 Jul 06 20:05:53.764670 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.764635 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:53.871789 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:53.871752 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:54.000999 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:54.000956 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" event={"ID":"41159632-e878-48ec-958d-7df6faa3ac60","Type":"ContainerStarted","Data":"4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04"} Jul 06 20:05:54.001167 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:54.001011 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" event={"ID":"41159632-e878-48ec-958d-7df6faa3ac60","Type":"ContainerStarted","Data":"22582b0ae20ecfb5e18eb384b1b8eb548be99e3a477e9deb64c58b79101da682"} Jul 06 20:05:54.001167 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:54.001049 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:05:54.027461 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:54.027354 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" podStartSLOduration=1.57898037 podStartE2EDuration="2.02733331s" totalImagesPullingTime="448.35294ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:05:54.02608303 +0000 UTC m=+2464.531368070" watchObservedRunningTime="2026-07-06 20:05:54.02733331 +0000 UTC m=+2464.532618347" Jul 06 20:05:57.236553 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:57.236514 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:57.569000 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:57.568888 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:57.950522 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:57.950425 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:05:58.255183 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:05:58.255141 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:00.011801 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:00.011768 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:06:00.785598 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:00.785553 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:00.906255 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:00.906217 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn"] Jul 06 20:06:00.910169 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:00.910141 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:00.912784 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:00.912758 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-ebe4df-serving-cert\"" Jul 06 20:06:00.927010 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:00.926979 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn"] Jul 06 20:06:00.972953 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:00.972888 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lplrz\" (UniqueName: \"kubernetes.io/projected/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-kube-api-access-lplrz\") pod \"maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:00.973150 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:00.972966 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:00.973150 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:00.973017 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-maas-api-tls\") pod \"maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:00.974546 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:00.974514 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:01.074204 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.074116 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:01.074204 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.074189 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-maas-api-tls\") pod \"maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:01.074606 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.074294 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lplrz\" (UniqueName: \"kubernetes.io/projected/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-kube-api-access-lplrz\") pod \"maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:01.074874 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.074853 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:01.076619 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.076594 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-maas-api-tls\") pod \"maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:01.086544 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.086476 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lplrz\" (UniqueName: \"kubernetes.io/projected/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-kube-api-access-lplrz\") pod \"maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:01.087830 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.087803 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:01.194365 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.194326 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:01.222972 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.222894 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:01.385720 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.385696 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn"] Jul 06 20:06:01.388106 ip-10-0-131-35 kubenswrapper[2572]: W0706 20:06:01.388081 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b06ea3_bcd9_4b11_ae3c_ecb0992bd914.slice/crio-433bc3813c9d79e3a4a066887b1f0f2d058a5fae4ce18917b731f9665072907e WatchSource:0}: Error finding container 433bc3813c9d79e3a4a066887b1f0f2d058a5fae4ce18917b731f9665072907e: Status 404 returned error can't find the container with id 433bc3813c9d79e3a4a066887b1f0f2d058a5fae4ce18917b731f9665072907e Jul 06 20:06:01.725174 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.725131 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:01.847059 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:01.847029 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:02.045395 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:02.045300 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" event={"ID":"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914","Type":"ContainerStarted","Data":"94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034"} Jul 06 20:06:02.045395 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:02.045370 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" event={"ID":"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914","Type":"ContainerStarted","Data":"433bc3813c9d79e3a4a066887b1f0f2d058a5fae4ce18917b731f9665072907e"} Jul 06 20:06:02.045688 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:02.045471 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:02.077822 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:02.077754 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" podStartSLOduration=1.68557381 podStartE2EDuration="2.07773275s" totalImagesPullingTime="392.15894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:02.076848626 +0000 UTC m=+2472.582133663" watchObservedRunningTime="2026-07-06 20:06:02.07773275 +0000 UTC m=+2472.583017789" Jul 06 20:06:05.233965 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:05.233924 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:08.056401 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:08.056370 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:06:12.962346 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:12.962295 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:14.841294 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:14.841254 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:14.958814 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:14.958773 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:22.331484 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:22.331445 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:22.428128 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:22.428089 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:22.669299 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:22.669212 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:22.783863 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:22.783828 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:35.753538 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:35.753496 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:35.881491 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:35.881449 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:43.718456 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.718357 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5"] Jul 06 20:06:43.722441 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.722419 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.726994 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.726834 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"kube-root-ca.crt\"" Jul 06 20:06:43.726994 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.726969 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"auth-test-model-ebe4df-kserve-self-signed-certs\"" Jul 06 20:06:43.727196 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.726976 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"openshift-service-ca.crt\"" Jul 06 20:06:43.727334 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.727318 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"default-dockercfg-gnj7g\"" Jul 06 20:06:43.734502 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.734477 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5"] Jul 06 20:06:43.779650 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.779611 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:43.782676 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.782651 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-dshm\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.782816 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.782722 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-model-cache\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.782816 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.782764 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76wf5\" (UniqueName: \"kubernetes.io/projected/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-kube-api-access-76wf5\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.782940 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.782836 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-home\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.782940 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.782855 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tmp-dir\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.782940 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.782870 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tls-certs\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.884341 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.884301 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-model-cache\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.884341 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.884343 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76wf5\" (UniqueName: \"kubernetes.io/projected/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-kube-api-access-76wf5\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.884600 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.884481 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-home\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.884600 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.884524 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tmp-dir\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.884600 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.884574 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tls-certs\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.884755 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.884679 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-dshm\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.884810 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.884788 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-model-cache\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.884846 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.884800 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tmp-dir\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.884942 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.884882 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-home\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.886957 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.886928 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-dshm\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.887198 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.887183 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tls-certs\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:43.898119 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:43.898090 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76wf5\" (UniqueName: \"kubernetes.io/projected/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-kube-api-access-76wf5\") pod \"auth-test-model-ebe4df-kserve-695db7f987-x4fj5\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:44.034588 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:44.034491 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:44.191963 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:44.191938 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5"] Jul 06 20:06:44.193965 ip-10-0-131-35 kubenswrapper[2572]: W0706 20:06:44.193937 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61e2d2c2_b921_49eb_9d10_c42f3a19bf72.slice/crio-0faf609fc4fdec3e2423f07d1725608fdcfcc732c77386bcf34763e62f3f93c6 WatchSource:0}: Error finding container 0faf609fc4fdec3e2423f07d1725608fdcfcc732c77386bcf34763e62f3f93c6: Status 404 returned error can't find the container with id 0faf609fc4fdec3e2423f07d1725608fdcfcc732c77386bcf34763e62f3f93c6 Jul 06 20:06:44.266621 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:44.266592 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" event={"ID":"61e2d2c2-b921-49eb-9d10-c42f3a19bf72","Type":"ContainerStarted","Data":"0faf609fc4fdec3e2423f07d1725608fdcfcc732c77386bcf34763e62f3f93c6"} Jul 06 20:06:45.272880 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:45.272839 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" event={"ID":"61e2d2c2-b921-49eb-9d10-c42f3a19bf72","Type":"ContainerStarted","Data":"65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6"} Jul 06 20:06:45.292705 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:45.292622 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" podStartSLOduration=2.29260592 podStartE2EDuration="2.29260592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:45.292006914 +0000 UTC m=+2515.797291953" watchObservedRunningTime="2026-07-06 20:06:45.29260592 +0000 UTC m=+2515.797890959" Jul 06 20:06:51.098356 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:51.098269 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:06:54.035642 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:54.035597 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:54.036237 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:54.035673 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:54.049069 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:54.049034 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:06:54.328597 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:06:54.328500 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:07:46.033605 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:07:46.033564 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:07:46.128484 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:07:46.128444 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:08:45.442388 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:08:45.442348 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:08:46.382010 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:08:46.381964 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5"] Jul 06 20:08:46.410662 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:08:46.410627 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:08:57.354990 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:08:57.354949 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:09:01.402573 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.402527 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" podUID="61e2d2c2-b921-49eb-9d10-c42f3a19bf72" containerName="main" containerID="cri-o://65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6" gracePeriod=45 Jul 06 20:09:01.662023 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.661954 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:09:01.686116 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.686086 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tmp-dir\") pod \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " Jul 06 20:09:01.686292 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.686144 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-home\") pod \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " Jul 06 20:09:01.686292 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.686186 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76wf5\" (UniqueName: \"kubernetes.io/projected/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-kube-api-access-76wf5\") pod \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " Jul 06 20:09:01.686292 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.686277 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-model-cache\") pod \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " Jul 06 20:09:01.686451 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.686306 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tls-certs\") pod \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " Jul 06 20:09:01.686451 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.686344 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-dshm\") pod \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\" (UID: \"61e2d2c2-b921-49eb-9d10-c42f3a19bf72\") " Jul 06 20:09:01.686451 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.686386 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-home" (OuterVolumeSpecName: "home") pod "61e2d2c2-b921-49eb-9d10-c42f3a19bf72" (UID: "61e2d2c2-b921-49eb-9d10-c42f3a19bf72"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:01.686451 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.686396 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "61e2d2c2-b921-49eb-9d10-c42f3a19bf72" (UID: "61e2d2c2-b921-49eb-9d10-c42f3a19bf72"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:01.686639 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.686572 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-model-cache" (OuterVolumeSpecName: "model-cache") pod "61e2d2c2-b921-49eb-9d10-c42f3a19bf72" (UID: "61e2d2c2-b921-49eb-9d10-c42f3a19bf72"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:01.686693 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.686641 2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tmp-dir\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:09:01.686693 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.686658 2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-home\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:09:01.688785 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.688755 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-dshm" (OuterVolumeSpecName: "dshm") pod "61e2d2c2-b921-49eb-9d10-c42f3a19bf72" (UID: "61e2d2c2-b921-49eb-9d10-c42f3a19bf72"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:01.688954 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.688802 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-kube-api-access-76wf5" (OuterVolumeSpecName: "kube-api-access-76wf5") pod "61e2d2c2-b921-49eb-9d10-c42f3a19bf72" (UID: "61e2d2c2-b921-49eb-9d10-c42f3a19bf72"). InnerVolumeSpecName "kube-api-access-76wf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:09:01.688954 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.688898 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "61e2d2c2-b921-49eb-9d10-c42f3a19bf72" (UID: "61e2d2c2-b921-49eb-9d10-c42f3a19bf72"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:09:01.788116 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.788080 2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-dshm\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:09:01.788116 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.788114 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76wf5\" (UniqueName: \"kubernetes.io/projected/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-kube-api-access-76wf5\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:09:01.788116 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.788125 2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-model-cache\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:09:01.788359 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.788134 2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61e2d2c2-b921-49eb-9d10-c42f3a19bf72-tls-certs\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:09:01.973051 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.973018 2572 generic.go:356] "Generic (PLEG): container finished" podID="61e2d2c2-b921-49eb-9d10-c42f3a19bf72" containerID="65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6" exitCode=0 Jul 06 20:09:01.973215 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.973064 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" event={"ID":"61e2d2c2-b921-49eb-9d10-c42f3a19bf72","Type":"ContainerDied","Data":"65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6"} Jul 06 20:09:01.973215 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.973092 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" event={"ID":"61e2d2c2-b921-49eb-9d10-c42f3a19bf72","Type":"ContainerDied","Data":"0faf609fc4fdec3e2423f07d1725608fdcfcc732c77386bcf34763e62f3f93c6"} Jul 06 20:09:01.973215 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.973108 2572 scope.go:117] "RemoveContainer" containerID="65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6" Jul 06 20:09:01.973215 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.973066 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5" Jul 06 20:09:01.982661 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.982640 2572 scope.go:117] "RemoveContainer" containerID="65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6" Jul 06 20:09:01.982948 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:09:01.982892 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6\": container with ID starting with 65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6 not found: ID does not exist" containerID="65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6" Jul 06 20:09:01.983059 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.982947 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6"} err="failed to get container status \"65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6\": rpc error: code = NotFound desc = could not find container \"65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6\": container with ID starting with 65742105e3cd4b3100107a447399ca77d1fe836e96e3e71da8b694c5c6c1c4f6 not found: ID does not exist" Jul 06 20:09:01.996612 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:01.996582 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5"] Jul 06 20:09:02.000338 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:02.000312 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/auth-test-model-ebe4df-kserve-695db7f987-x4fj5"] Jul 06 20:09:02.176221 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:02.176188 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61e2d2c2-b921-49eb-9d10-c42f3a19bf72" path="/var/lib/kubelet/pods/61e2d2c2-b921-49eb-9d10-c42f3a19bf72/volumes" Jul 06 20:09:22.384784 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.384745 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v"] Jul 06 20:09:22.385604 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.385163 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61e2d2c2-b921-49eb-9d10-c42f3a19bf72" containerName="main" Jul 06 20:09:22.385604 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.385177 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="61e2d2c2-b921-49eb-9d10-c42f3a19bf72" containerName="main" Jul 06 20:09:22.385604 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.385258 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="61e2d2c2-b921-49eb-9d10-c42f3a19bf72" containerName="main" Jul 06 20:09:22.389789 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.389766 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.392394 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.392372 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"openshift-service-ca.crt\"" Jul 06 20:09:22.393291 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.393270 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"kube-root-ca.crt\"" Jul 06 20:09:22.393382 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.393360 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"sub-test-model-ebe4df-kserve-self-signed-certs\"" Jul 06 20:09:22.393426 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.393379 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"default-dockercfg-gnj7g\"" Jul 06 20:09:22.399133 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.399104 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v"] Jul 06 20:09:22.418171 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.418133 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:09:22.480178 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.480121 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-home\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.480178 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.480174 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tls-certs\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.480416 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.480211 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rm8s\" (UniqueName: \"kubernetes.io/projected/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-kube-api-access-7rm8s\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.480416 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.480271 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tmp-dir\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.480416 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.480338 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-dshm\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.480416 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.480404 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-model-cache\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.581425 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.581369 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-model-cache\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.581425 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.581438 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-home\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.581706 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.581459 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tls-certs\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.581706 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.581477 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rm8s\" (UniqueName: \"kubernetes.io/projected/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-kube-api-access-7rm8s\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.581706 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.581542 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tmp-dir\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.581706 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.581586 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-dshm\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.582068 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.582042 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-home\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.582173 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.582051 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-model-cache\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.582173 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.582112 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tmp-dir\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.583928 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.583894 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-dshm\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.584220 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.584204 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tls-certs\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.592755 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.592733 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rm8s\" (UniqueName: \"kubernetes.io/projected/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-kube-api-access-7rm8s\") pod \"sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.699814 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.699722 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:22.850572 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:22.850539 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v"] Jul 06 20:09:22.851241 ip-10-0-131-35 kubenswrapper[2572]: W0706 20:09:22.851213 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3c7b5d7_bec2_4153_9762_7f4b883c9ec6.slice/crio-75f8a7e3f5d9d00f6366829a4befbebe24dbdaac663a61b7da3d33e69da6924c WatchSource:0}: Error finding container 75f8a7e3f5d9d00f6366829a4befbebe24dbdaac663a61b7da3d33e69da6924c: Status 404 returned error can't find the container with id 75f8a7e3f5d9d00f6366829a4befbebe24dbdaac663a61b7da3d33e69da6924c Jul 06 20:09:23.079057 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:23.079017 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" event={"ID":"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6","Type":"ContainerStarted","Data":"92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f"} Jul 06 20:09:23.079233 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:23.079068 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" event={"ID":"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6","Type":"ContainerStarted","Data":"75f8a7e3f5d9d00f6366829a4befbebe24dbdaac663a61b7da3d33e69da6924c"} Jul 06 20:09:23.099189 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:23.099136 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" podStartSLOduration=1.099118449 podStartE2EDuration="1.099118449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:23.098003806 +0000 UTC m=+2673.603288846" watchObservedRunningTime="2026-07-06 20:09:23.099118449 +0000 UTC m=+2673.604403541" Jul 06 20:09:32.700839 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:32.700798 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:32.701413 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:32.700887 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:32.713390 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:32.713365 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:33.145674 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:33.145637 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:09:50.147539 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:50.147440 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 20:09:50.151840 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:50.151815 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 20:09:50.155501 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:50.155477 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 20:09:50.159840 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:09:50.159821 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5d4pt_cbf09284-56b5-453a-8ed0-c33a09b04265/ovn-acl-logging/0.log" Jul 06 20:10:12.680192 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:12.680151 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:10:13.614698 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:13.614659 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v"] Jul 06 20:10:13.946454 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:13.946351 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:10:17.065121 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:17.065083 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:10:28.635415 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:28.635365 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" podUID="f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" containerName="main" containerID="cri-o://92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f" gracePeriod=45 Jul 06 20:10:28.899419 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:28.899349 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:10:29.025869 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.025832 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-home\") pod \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " Jul 06 20:10:29.025869 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.025874 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rm8s\" (UniqueName: \"kubernetes.io/projected/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-kube-api-access-7rm8s\") pod \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " Jul 06 20:10:29.026124 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.025930 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-dshm\") pod \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " Jul 06 20:10:29.026124 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.025964 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-model-cache\") pod \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " Jul 06 20:10:29.026124 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.026098 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tmp-dir\") pod \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " Jul 06 20:10:29.026266 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.026130 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tls-certs\") pod \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\" (UID: \"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6\") " Jul 06 20:10:29.026266 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.026202 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-home" (OuterVolumeSpecName: "home") pod "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" (UID: "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:10:29.026379 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.026307 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" (UID: "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:10:29.026467 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.026416 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" (UID: "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:10:29.026587 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.026518 2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-home\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:10:29.026587 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.026538 2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-model-cache\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:10:29.028125 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.028096 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-dshm" (OuterVolumeSpecName: "dshm") pod "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" (UID: "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:10:29.028236 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.028158 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" (UID: "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:10:29.028236 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.028207 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-kube-api-access-7rm8s" (OuterVolumeSpecName: "kube-api-access-7rm8s") pod "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" (UID: "f3c7b5d7-bec2-4153-9762-7f4b883c9ec6"). InnerVolumeSpecName "kube-api-access-7rm8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:10:29.127251 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.127213 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rm8s\" (UniqueName: \"kubernetes.io/projected/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-kube-api-access-7rm8s\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:10:29.127251 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.127244 2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-dshm\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:10:29.127251 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.127256 2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tmp-dir\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:10:29.127251 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.127264 2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6-tls-certs\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:10:29.427367 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.427334 2572 generic.go:356] "Generic (PLEG): container finished" podID="f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" containerID="92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f" exitCode=0 Jul 06 20:10:29.427542 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.427375 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" Jul 06 20:10:29.427542 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.427367 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" event={"ID":"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6","Type":"ContainerDied","Data":"92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f"} Jul 06 20:10:29.427542 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.427475 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v" event={"ID":"f3c7b5d7-bec2-4153-9762-7f4b883c9ec6","Type":"ContainerDied","Data":"75f8a7e3f5d9d00f6366829a4befbebe24dbdaac663a61b7da3d33e69da6924c"} Jul 06 20:10:29.427542 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.427492 2572 scope.go:117] "RemoveContainer" containerID="92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f" Jul 06 20:10:29.438001 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.437967 2572 scope.go:117] "RemoveContainer" containerID="92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f" Jul 06 20:10:29.438299 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:10:29.438280 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f\": container with ID starting with 92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f not found: ID does not exist" containerID="92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f" Jul 06 20:10:29.438377 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.438312 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f"} err="failed to get container status \"92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f\": rpc error: code = NotFound desc = could not find container \"92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f\": container with ID starting with 92ff00a79e68711de9b23392c33c4abdfa7f0519aa532866510fb86c17ea204f not found: ID does not exist" Jul 06 20:10:29.456538 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.456504 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v"] Jul 06 20:10:29.464820 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:29.464786 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/sub-test-model-ebe4df-kserve-c9d9c66d7-p4j9v"] Jul 06 20:10:30.176722 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:30.176687 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" path="/var/lib/kubelet/pods/f3c7b5d7-bec2-4153-9762-7f4b883c9ec6/volumes" Jul 06 20:10:33.531335 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.531300 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g"] Jul 06 20:10:33.531719 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.531685 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" containerName="main" Jul 06 20:10:33.531719 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.531695 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" containerName="main" Jul 06 20:10:33.531798 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.531759 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3c7b5d7-bec2-4153-9762-7f4b883c9ec6" containerName="main" Jul 06 20:10:33.536131 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.536101 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.540127 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.540096 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"kube-root-ca.crt\"" Jul 06 20:10:33.540315 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.540149 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"openshift-service-ca.crt\"" Jul 06 20:10:33.540315 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.540097 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"rate-test-model-ebe4df-kserve-self-signed-certs\"" Jul 06 20:10:33.540315 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.540234 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ebe4df\"/\"default-dockercfg-gnj7g\"" Jul 06 20:10:33.564952 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.564901 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g"] Jul 06 20:10:33.599950 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.599888 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:10:33.670204 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.670157 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tls-certs\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.670419 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.670229 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-dshm\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.670419 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.670272 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-model-cache\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.670419 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.670297 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbkpb\" (UniqueName: \"kubernetes.io/projected/c1e655c9-981e-4302-ab91-32cfb9d02fe0-kube-api-access-vbkpb\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.670818 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.670520 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-home\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.671069 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.671046 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tmp-dir\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.771780 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.771740 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-dshm\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.771780 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.771789 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-model-cache\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.772097 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.771809 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbkpb\" (UniqueName: \"kubernetes.io/projected/c1e655c9-981e-4302-ab91-32cfb9d02fe0-kube-api-access-vbkpb\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.772097 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.771857 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-home\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.772097 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.771880 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tmp-dir\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.772097 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.771996 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tls-certs\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.772323 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.772305 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-home\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.772385 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.772366 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-model-cache\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.772385 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.772376 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tmp-dir\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.774191 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.774168 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-dshm\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.774439 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.774422 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tls-certs\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.793759 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.793674 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbkpb\" (UniqueName: \"kubernetes.io/projected/c1e655c9-981e-4302-ab91-32cfb9d02fe0-kube-api-access-vbkpb\") pod \"rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:33.846589 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:33.846549 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:34.002238 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:34.002209 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g"] Jul 06 20:10:34.004569 ip-10-0-131-35 kubenswrapper[2572]: W0706 20:10:34.004541 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1e655c9_981e_4302_ab91_32cfb9d02fe0.slice/crio-9c1ab42b2df8fa57f77490ac949324913e0b60775291426c5f26d08f54457f09 WatchSource:0}: Error finding container 9c1ab42b2df8fa57f77490ac949324913e0b60775291426c5f26d08f54457f09: Status 404 returned error can't find the container with id 9c1ab42b2df8fa57f77490ac949324913e0b60775291426c5f26d08f54457f09 Jul 06 20:10:34.457754 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:34.457657 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" event={"ID":"c1e655c9-981e-4302-ab91-32cfb9d02fe0","Type":"ContainerStarted","Data":"99217e59235ff20e54e4b241b09ad81e1ec1fdca9ebd9888ba69cd64b2597d8a"} Jul 06 20:10:34.457754 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:34.457712 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" event={"ID":"c1e655c9-981e-4302-ab91-32cfb9d02fe0","Type":"ContainerStarted","Data":"9c1ab42b2df8fa57f77490ac949324913e0b60775291426c5f26d08f54457f09"} Jul 06 20:10:34.483776 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:34.483721 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" podStartSLOduration=1.483706046 podStartE2EDuration="1.483706046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:10:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:10:34.481499101 +0000 UTC m=+2744.986784140" watchObservedRunningTime="2026-07-06 20:10:34.483706046 +0000 UTC m=+2744.988991104" Jul 06 20:10:43.847359 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:43.847306 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:43.847753 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:43.847381 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:43.860407 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:43.860376 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:10:44.519497 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:10:44.519470 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:11:15.760694 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:15.760650 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:15.863436 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:15.863395 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:22.613009 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:22.612971 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:23.508353 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:23.508321 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:23.520370 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:23.520328 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g"] Jul 06 20:11:23.664317 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:23.664276 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:24.689636 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:24.689591 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:25.065153 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.065114 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:25.342415 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.342313 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:25.436685 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.436650 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:25.650440 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.650346 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8"] Jul 06 20:11:25.650771 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.650690 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" podUID="41159632-e878-48ec-958d-7df6faa3ac60" containerName="maas-api" containerID="cri-o://4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04" gracePeriod=30 Jul 06 20:11:25.681021 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.680984 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:25.903246 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.903177 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:11:25.957132 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.957096 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41159632-e878-48ec-958d-7df6faa3ac60-maas-api-tls\") pod \"41159632-e878-48ec-958d-7df6faa3ac60\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " Jul 06 20:11:25.957328 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.957232 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41159632-e878-48ec-958d-7df6faa3ac60-openshift-service-ca\") pod \"41159632-e878-48ec-958d-7df6faa3ac60\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " Jul 06 20:11:25.957328 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.957273 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9pmx\" (UniqueName: \"kubernetes.io/projected/41159632-e878-48ec-958d-7df6faa3ac60-kube-api-access-l9pmx\") pod \"41159632-e878-48ec-958d-7df6faa3ac60\" (UID: \"41159632-e878-48ec-958d-7df6faa3ac60\") " Jul 06 20:11:25.957584 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.957550 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41159632-e878-48ec-958d-7df6faa3ac60-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "41159632-e878-48ec-958d-7df6faa3ac60" (UID: "41159632-e878-48ec-958d-7df6faa3ac60"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:11:25.959480 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.959443 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41159632-e878-48ec-958d-7df6faa3ac60-kube-api-access-l9pmx" (OuterVolumeSpecName: "kube-api-access-l9pmx") pod "41159632-e878-48ec-958d-7df6faa3ac60" (UID: "41159632-e878-48ec-958d-7df6faa3ac60"). InnerVolumeSpecName "kube-api-access-l9pmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:11:25.959587 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:25.959450 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41159632-e878-48ec-958d-7df6faa3ac60-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "41159632-e878-48ec-958d-7df6faa3ac60" (UID: "41159632-e878-48ec-958d-7df6faa3ac60"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:26.058782 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.058737 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9pmx\" (UniqueName: \"kubernetes.io/projected/41159632-e878-48ec-958d-7df6faa3ac60-kube-api-access-l9pmx\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:26.058782 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.058773 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41159632-e878-48ec-958d-7df6faa3ac60-maas-api-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:26.058782 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.058783 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41159632-e878-48ec-958d-7df6faa3ac60-openshift-service-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:26.733437 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.733405 2572 generic.go:356] "Generic (PLEG): container finished" podID="41159632-e878-48ec-958d-7df6faa3ac60" containerID="4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04" exitCode=0 Jul 06 20:11:26.733641 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.733446 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" event={"ID":"41159632-e878-48ec-958d-7df6faa3ac60","Type":"ContainerDied","Data":"4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04"} Jul 06 20:11:26.733641 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.733484 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" event={"ID":"41159632-e878-48ec-958d-7df6faa3ac60","Type":"ContainerDied","Data":"22582b0ae20ecfb5e18eb384b1b8eb548be99e3a477e9deb64c58b79101da682"} Jul 06 20:11:26.733641 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.733507 2572 scope.go:117] "RemoveContainer" containerID="4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04" Jul 06 20:11:26.733641 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.733448 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8" Jul 06 20:11:26.742822 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.742800 2572 scope.go:117] "RemoveContainer" containerID="4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04" Jul 06 20:11:26.743166 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:11:26.743145 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04\": container with ID starting with 4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04 not found: ID does not exist" containerID="4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04" Jul 06 20:11:26.743227 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.743176 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04"} err="failed to get container status \"4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04\": rpc error: code = NotFound desc = could not find container \"4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04\": container with ID starting with 4b065cf4b155c26a5be8176f659e4f1a5638f010fb4e7c9b5b6efb0156714c04 not found: ID does not exist" Jul 06 20:11:26.755476 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.755443 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8"] Jul 06 20:11:26.760547 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:26.760521 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-7ba461-858dcf57bc-n8bd8"] Jul 06 20:11:28.175547 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:28.175510 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41159632-e878-48ec-958d-7df6faa3ac60" path="/var/lib/kubelet/pods/41159632-e878-48ec-958d-7df6faa3ac60/volumes" Jul 06 20:11:30.122311 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:30.122271 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:36.713194 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:36.713156 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:36.810144 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:36.810098 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:38.543858 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.543808 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" podUID="c1e655c9-981e-4302-ab91-32cfb9d02fe0" containerName="main" containerID="cri-o://99217e59235ff20e54e4b241b09ad81e1ec1fdca9ebd9888ba69cd64b2597d8a" gracePeriod=45 Jul 06 20:11:38.802795 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.802732 2572 generic.go:356] "Generic (PLEG): container finished" podID="c1e655c9-981e-4302-ab91-32cfb9d02fe0" containerID="99217e59235ff20e54e4b241b09ad81e1ec1fdca9ebd9888ba69cd64b2597d8a" exitCode=0 Jul 06 20:11:38.802941 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.802783 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" event={"ID":"c1e655c9-981e-4302-ab91-32cfb9d02fe0","Type":"ContainerDied","Data":"99217e59235ff20e54e4b241b09ad81e1ec1fdca9ebd9888ba69cd64b2597d8a"} Jul 06 20:11:38.818959 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.818931 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:11:38.982096 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982053 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-home\") pod \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " Jul 06 20:11:38.982096 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982103 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tmp-dir\") pod \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " Jul 06 20:11:38.982370 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982145 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-model-cache\") pod \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " Jul 06 20:11:38.982370 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982168 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbkpb\" (UniqueName: \"kubernetes.io/projected/c1e655c9-981e-4302-ab91-32cfb9d02fe0-kube-api-access-vbkpb\") pod \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " Jul 06 20:11:38.982370 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982214 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-dshm\") pod \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " Jul 06 20:11:38.982370 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982292 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tls-certs\") pod \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\" (UID: \"c1e655c9-981e-4302-ab91-32cfb9d02fe0\") " Jul 06 20:11:38.982576 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982407 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-home" (OuterVolumeSpecName: "home") pod "c1e655c9-981e-4302-ab91-32cfb9d02fe0" (UID: "c1e655c9-981e-4302-ab91-32cfb9d02fe0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:11:38.982576 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982453 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-model-cache" (OuterVolumeSpecName: "model-cache") pod "c1e655c9-981e-4302-ab91-32cfb9d02fe0" (UID: "c1e655c9-981e-4302-ab91-32cfb9d02fe0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:11:38.982576 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982470 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c1e655c9-981e-4302-ab91-32cfb9d02fe0" (UID: "c1e655c9-981e-4302-ab91-32cfb9d02fe0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:11:38.982723 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982662 2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-home\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:38.982723 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982682 2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tmp-dir\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:38.982723 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.982697 2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-model-cache\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:38.984480 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.984447 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c1e655c9-981e-4302-ab91-32cfb9d02fe0" (UID: "c1e655c9-981e-4302-ab91-32cfb9d02fe0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:38.984584 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.984452 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-dshm" (OuterVolumeSpecName: "dshm") pod "c1e655c9-981e-4302-ab91-32cfb9d02fe0" (UID: "c1e655c9-981e-4302-ab91-32cfb9d02fe0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:11:38.984584 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:38.984505 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e655c9-981e-4302-ab91-32cfb9d02fe0-kube-api-access-vbkpb" (OuterVolumeSpecName: "kube-api-access-vbkpb") pod "c1e655c9-981e-4302-ab91-32cfb9d02fe0" (UID: "c1e655c9-981e-4302-ab91-32cfb9d02fe0"). InnerVolumeSpecName "kube-api-access-vbkpb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:11:39.084193 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:39.084103 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbkpb\" (UniqueName: \"kubernetes.io/projected/c1e655c9-981e-4302-ab91-32cfb9d02fe0-kube-api-access-vbkpb\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:39.084193 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:39.084132 2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1e655c9-981e-4302-ab91-32cfb9d02fe0-dshm\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:39.084193 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:39.084143 2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1e655c9-981e-4302-ab91-32cfb9d02fe0-tls-certs\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:39.810081 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:39.810048 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" Jul 06 20:11:39.810081 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:39.810059 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g" event={"ID":"c1e655c9-981e-4302-ab91-32cfb9d02fe0","Type":"ContainerDied","Data":"9c1ab42b2df8fa57f77490ac949324913e0b60775291426c5f26d08f54457f09"} Jul 06 20:11:39.810644 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:39.810106 2572 scope.go:117] "RemoveContainer" containerID="99217e59235ff20e54e4b241b09ad81e1ec1fdca9ebd9888ba69cd64b2597d8a" Jul 06 20:11:39.843212 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:39.843176 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g"] Jul 06 20:11:39.855136 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:39.855099 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ebe4df/rate-test-model-ebe4df-kserve-748b9d9bf5-gwt7g"] Jul 06 20:11:40.176318 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:40.176235 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1e655c9-981e-4302-ab91-32cfb9d02fe0" path="/var/lib/kubelet/pods/c1e655c9-981e-4302-ab91-32cfb9d02fe0/volumes" Jul 06 20:11:43.560333 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:43.560266 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:43.823940 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:43.823799 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:44.064543 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:44.064507 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:44.246527 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:44.246486 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:44.701448 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:44.701360 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:44.804225 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:44.804189 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:44.901863 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:44.901815 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:45.068347 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.068310 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn"] Jul 06 20:11:45.068586 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.068562 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" podUID="51b06ea3-bcd9-4b11-ae3c-ecb0992bd914" containerName="maas-api" containerID="cri-o://94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034" gracePeriod=30 Jul 06 20:11:45.094495 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.094460 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:45.320169 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.320096 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:11:45.445638 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.445603 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-openshift-service-ca\") pod \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " Jul 06 20:11:45.445794 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.445651 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-maas-api-tls\") pod \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " Jul 06 20:11:45.445794 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.445703 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lplrz\" (UniqueName: \"kubernetes.io/projected/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-kube-api-access-lplrz\") pod \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\" (UID: \"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914\") " Jul 06 20:11:45.446101 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.446066 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "51b06ea3-bcd9-4b11-ae3c-ecb0992bd914" (UID: "51b06ea3-bcd9-4b11-ae3c-ecb0992bd914"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:11:45.447964 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.447931 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-kube-api-access-lplrz" (OuterVolumeSpecName: "kube-api-access-lplrz") pod "51b06ea3-bcd9-4b11-ae3c-ecb0992bd914" (UID: "51b06ea3-bcd9-4b11-ae3c-ecb0992bd914"). InnerVolumeSpecName "kube-api-access-lplrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:11:45.448102 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.447991 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "51b06ea3-bcd9-4b11-ae3c-ecb0992bd914" (UID: "51b06ea3-bcd9-4b11-ae3c-ecb0992bd914"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:45.547078 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.547043 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lplrz\" (UniqueName: \"kubernetes.io/projected/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-kube-api-access-lplrz\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:45.547078 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.547073 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-openshift-service-ca\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:45.547078 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.547083 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914-maas-api-tls\") on node \"ip-10-0-131-35.ec2.internal\" DevicePath \"\"" Jul 06 20:11:45.843403 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.843364 2572 generic.go:356] "Generic (PLEG): container finished" podID="51b06ea3-bcd9-4b11-ae3c-ecb0992bd914" containerID="94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034" exitCode=0 Jul 06 20:11:45.843403 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.843405 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" Jul 06 20:11:45.843940 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.843426 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" event={"ID":"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914","Type":"ContainerDied","Data":"94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034"} Jul 06 20:11:45.843940 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.843482 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn" event={"ID":"51b06ea3-bcd9-4b11-ae3c-ecb0992bd914","Type":"ContainerDied","Data":"433bc3813c9d79e3a4a066887b1f0f2d058a5fae4ce18917b731f9665072907e"} Jul 06 20:11:45.843940 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.843512 2572 scope.go:117] "RemoveContainer" containerID="94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034" Jul 06 20:11:45.854146 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.854121 2572 scope.go:117] "RemoveContainer" containerID="94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034" Jul 06 20:11:45.854453 ip-10-0-131-35 kubenswrapper[2572]: E0706 20:11:45.854433 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034\": container with ID starting with 94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034 not found: ID does not exist" containerID="94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034" Jul 06 20:11:45.854498 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.854466 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034"} err="failed to get container status \"94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034\": rpc error: code = NotFound desc = could not find container \"94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034\": container with ID starting with 94729a0ac9d32de958e279a15f2fe25d42c04224cd297de692611d27392ca034 not found: ID does not exist" Jul 06 20:11:45.870467 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.870434 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn"] Jul 06 20:11:45.875649 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:45.875618 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-ebe4df-9475d7cc-hzxfn"] Jul 06 20:11:46.176208 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:46.176127 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51b06ea3-bcd9-4b11-ae3c-ecb0992bd914" path="/var/lib/kubelet/pods/51b06ea3-bcd9-4b11-ae3c-ecb0992bd914/volumes" Jul 06 20:11:51.290427 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:51.290392 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:51.363661 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:51.363607 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:51.561168 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:51.561070 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:51.780158 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:51.780116 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:11:56.817002 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:56.816970 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lwftm_fad8e3b0-82a9-4d95-8807-aa3819bcb9e7/manager/1.log" Jul 06 20:11:56.935859 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:56.935818 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-x2577_8f8ff396-377b-4cd6-a4f2-9b26caeab604/manager/1.log" Jul 06 20:11:57.058993 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:57.058954 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-b6cbd65b-lhch9_1d397eeb-06d2-4a41-9518-a974b9c64d03/maas-api/0.log" Jul 06 20:11:57.991075 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:57.991044 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849ccdb5-xh49b_8bf46798-5ef5-4e66-b54a-5928af921668/copy-manifests/0.log" Jul 06 20:11:58.010752 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:58.010712 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849ccdb5-xh49b_8bf46798-5ef5-4e66-b54a-5928af921668/manager/1.log" Jul 06 20:11:59.625163 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:59.625137 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8jm6m_233283df-b179-4063-97ec-96608510f7d7/manager/1.log" Jul 06 20:11:59.979228 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:11:59.979199 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-49k9p_bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa/manager/1.log" Jul 06 20:12:01.159527 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:01.159491 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-fdtsv_a1405983-5015-401d-a880-cb4abbd175f8/payload-pre-processing/0.log" Jul 06 20:12:01.392106 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:01.392071 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-756fc79d99-fvh2j_2369af8d-444e-488a-97c3-692ee5138a01/router/0.log" Jul 06 20:12:01.846357 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:01.846323 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-bxnf2_ba17bcd2-f7a7-4966-a57a-4371830a6346/main/0.log" Jul 06 20:12:02.072502 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:02.072465 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh68l8_5386d35a-6af2-4cd1-affa-ed864ed8f9f9/main/0.log" Jul 06 20:12:02.295646 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:02.295617 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-wdcx8_26490728-c426-46f3-99ec-d0d165ef0999/main/0.log" Jul 06 20:12:06.016647 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:06.016614 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:12:06.116584 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:06.116545 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" limit=1024 Jul 06 20:12:09.129137 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:09.129105 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cjr7k_b2cbb8d1-a657-43ca-a8d1-bc09a049a710/global-pull-secret-syncer/0.log" Jul 06 20:12:09.275855 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:09.275817 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5pnv2_663a3e6c-13db-45ac-b7fd-0efd27a4b159/konnectivity-agent/0.log" Jul 06 20:12:09.401470 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:09.401379 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-35.ec2.internal_a91a71edc512a42ccb4fd1df84711f15/haproxy/0.log" Jul 06 20:12:14.341853 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:14.341813 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8jm6m_233283df-b179-4063-97ec-96608510f7d7/manager/1.log" Jul 06 20:12:14.342297 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:14.341992 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8jm6m_233283df-b179-4063-97ec-96608510f7d7/manager/0.log" Jul 06 20:12:14.521086 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:14.521041 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-49k9p_bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa/manager/1.log" Jul 06 20:12:14.597151 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:14.597065 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-49k9p_bcd638e2-a4ec-4427-b1c1-5a43dc4aa7fa/manager/0.log" Jul 06 20:12:16.686079 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:16.686040 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-lx4sd_55445c8d-30b3-47cd-9479-1ca223c726e5/cluster-monitoring-operator/0.log" Jul 06 20:12:17.042477 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:17.042446 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c2h4w_8beb7b2f-4c3e-493c-b734-c75fadc161ad/node-exporter/0.log" Jul 06 20:12:17.074369 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:17.074342 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c2h4w_8beb7b2f-4c3e-493c-b734-c75fadc161ad/kube-rbac-proxy/0.log" Jul 06 20:12:17.097497 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:17.097469 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c2h4w_8beb7b2f-4c3e-493c-b734-c75fadc161ad/init-textfile/0.log" Jul 06 20:12:18.966112 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:18.966090 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-fqdb4_94235910-e2b4-4f48-9c04-9a01df47b39e/networking-console-plugin/0.log" Jul 06 20:12:19.394289 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.394205 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr"] Jul 06 20:12:19.394571 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.394559 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41159632-e878-48ec-958d-7df6faa3ac60" containerName="maas-api" Jul 06 20:12:19.394614 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.394573 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="41159632-e878-48ec-958d-7df6faa3ac60" containerName="maas-api" Jul 06 20:12:19.394614 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.394588 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b06ea3-bcd9-4b11-ae3c-ecb0992bd914" containerName="maas-api" Jul 06 20:12:19.394614 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.394595 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b06ea3-bcd9-4b11-ae3c-ecb0992bd914" containerName="maas-api" Jul 06 20:12:19.394614 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.394602 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1e655c9-981e-4302-ab91-32cfb9d02fe0" containerName="main" Jul 06 20:12:19.394614 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.394608 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1e655c9-981e-4302-ab91-32cfb9d02fe0" containerName="main" Jul 06 20:12:19.394787 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.394665 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="41159632-e878-48ec-958d-7df6faa3ac60" containerName="maas-api" Jul 06 20:12:19.394787 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.394675 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1e655c9-981e-4302-ab91-32cfb9d02fe0" containerName="main" Jul 06 20:12:19.394787 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.394684 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="51b06ea3-bcd9-4b11-ae3c-ecb0992bd914" containerName="maas-api" Jul 06 20:12:19.397764 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.397743 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.400207 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.400184 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nfpj8\"/\"openshift-service-ca.crt\"" Jul 06 20:12:19.401113 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.401094 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nfpj8\"/\"default-dockercfg-jjzk2\"" Jul 06 20:12:19.401235 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.401135 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nfpj8\"/\"kube-root-ca.crt\"" Jul 06 20:12:19.408340 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.408313 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr"] Jul 06 20:12:19.545225 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.545186 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-podres\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.545410 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.545246 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-proc\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.545410 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.545342 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-sys\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.545410 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.545380 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7smt5\" (UniqueName: \"kubernetes.io/projected/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-kube-api-access-7smt5\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.545548 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.545425 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-lib-modules\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.644660 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.644568 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/1.log" Jul 06 20:12:19.645942 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.645883 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-lib-modules\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.646180 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.645984 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-podres\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.646180 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.646059 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-lib-modules\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.646180 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.646065 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-proc\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.646180 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.646120 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-proc\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.646417 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.646204 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-sys\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.646417 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.646247 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7smt5\" (UniqueName: \"kubernetes.io/projected/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-kube-api-access-7smt5\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.646417 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.646271 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-sys\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.646417 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.646205 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-podres\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.653615 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.653588 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6cqbs_5bd27dda-064e-48ba-8ae7-d5a14b609bd3/console-operator/2.log" Jul 06 20:12:19.658459 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.658436 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7smt5\" (UniqueName: \"kubernetes.io/projected/e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d-kube-api-access-7smt5\") pod \"perf-node-gather-daemonset-5crkr\" (UID: \"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d\") " pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.708502 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.708458 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:19.845358 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:19.845331 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr"] Jul 06 20:12:19.847717 ip-10-0-131-35 kubenswrapper[2572]: W0706 20:12:19.847678 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode3d66f5f_ef6f_4258_b79c_5ad8d6758e0d.slice/crio-116a3e96cce63e07c61d8b8d8dd28e06ce624b83fe57cdf311d181fdcdd34b42 WatchSource:0}: Error finding container 116a3e96cce63e07c61d8b8d8dd28e06ce624b83fe57cdf311d181fdcdd34b42: Status 404 returned error can't find the container with id 116a3e96cce63e07c61d8b8d8dd28e06ce624b83fe57cdf311d181fdcdd34b42 Jul 06 20:12:20.019743 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:20.019706 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" event={"ID":"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d","Type":"ContainerStarted","Data":"27eec65b14e904fa16868a0706b9a099c97fc465a23beba744a301bd1d71c93d"} Jul 06 20:12:20.020198 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:20.019759 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" event={"ID":"e3d66f5f-ef6f-4258-b79c-5ad8d6758e0d","Type":"ContainerStarted","Data":"116a3e96cce63e07c61d8b8d8dd28e06ce624b83fe57cdf311d181fdcdd34b42"} Jul 06 20:12:20.020198 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:20.019801 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:20.036999 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:20.036938 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" podStartSLOduration=1.036887611 podStartE2EDuration="1.036887611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:12:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:12:20.03655696 +0000 UTC m=+2850.541842011" watchObservedRunningTime="2026-07-06 20:12:20.036887611 +0000 UTC m=+2850.542172649" Jul 06 20:12:20.811003 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:20.810972 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-k6bbz_7f1c75a3-c731-4050-82ba-444e5f4d5a9a/volume-data-source-validator/0.log" Jul 06 20:12:21.958046 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:21.958013 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gd48l_752bf161-7c8b-4af8-9d5d-3a7f9260e693/dns/0.log" Jul 06 20:12:21.980339 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:21.980312 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gd48l_752bf161-7c8b-4af8-9d5d-3a7f9260e693/kube-rbac-proxy/0.log" Jul 06 20:12:22.031006 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:22.030970 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ljx8t_68f9a6e3-08f5-4aa1-aba6-f07ff314e006/dns-node-resolver/0.log" Jul 06 20:12:22.684536 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:22.684495 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z2r5s_09767752-33c4-4574-b852-5d40a6c8929f/node-ca/0.log" Jul 06 20:12:23.898810 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:23.898778 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-fdtsv_a1405983-5015-401d-a880-cb4abbd175f8/payload-pre-processing/0.log" Jul 06 20:12:23.965287 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:23.965258 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-756fc79d99-fvh2j_2369af8d-444e-488a-97c3-692ee5138a01/router/0.log" Jul 06 20:12:24.671795 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:24.671768 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hfr9w_4eee88bf-1be6-492f-82eb-b64464b0199e/serve-healthcheck-canary/0.log" Jul 06 20:12:25.167228 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:25.167190 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-rg9vj_239c95d1-5bdb-4078-ad6b-9eb593957bb6/insights-operator/0.log" Jul 06 20:12:25.169501 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:25.169474 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-rg9vj_239c95d1-5bdb-4078-ad6b-9eb593957bb6/insights-operator/1.log" Jul 06 20:12:25.376825 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:25.376797 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x6xzx_87921692-d802-479d-a041-a5ba5b5f78af/kube-rbac-proxy/0.log" Jul 06 20:12:25.400717 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:25.400689 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x6xzx_87921692-d802-479d-a041-a5ba5b5f78af/exporter/0.log" Jul 06 20:12:25.426341 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:25.426260 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x6xzx_87921692-d802-479d-a041-a5ba5b5f78af/extractor/0.log" Jul 06 20:12:26.034788 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:26.034758 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nfpj8/perf-node-gather-daemonset-5crkr" Jul 06 20:12:27.526545 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:27.526517 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lwftm_fad8e3b0-82a9-4d95-8807-aa3819bcb9e7/manager/1.log" Jul 06 20:12:27.527102 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:27.526672 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lwftm_fad8e3b0-82a9-4d95-8807-aa3819bcb9e7/manager/0.log" Jul 06 20:12:27.673975 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:27.673943 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-x2577_8f8ff396-377b-4cd6-a4f2-9b26caeab604/manager/1.log" Jul 06 20:12:27.693308 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:27.693272 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-x2577_8f8ff396-377b-4cd6-a4f2-9b26caeab604/manager/0.log" Jul 06 20:12:27.741769 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:27.741735 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-b6cbd65b-lhch9_1d397eeb-06d2-4a41-9518-a974b9c64d03/maas-api/0.log" Jul 06 20:12:28.166395 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:28.166349 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849ccdb5-xh49b_8bf46798-5ef5-4e66-b54a-5928af921668/manager/1.log" Jul 06 20:12:28.202313 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:28.202274 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849ccdb5-xh49b_8bf46798-5ef5-4e66-b54a-5928af921668/manager/0.log" Jul 06 20:12:28.220202 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:28.220171 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849ccdb5-xh49b_8bf46798-5ef5-4e66-b54a-5928af921668/copy-manifests/0.log" Jul 06 20:12:29.748709 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:29.748674 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-fdtsv_a1405983-5015-401d-a880-cb4abbd175f8/payload-pre-processing/0.log" Jul 06 20:12:29.778659 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:29.778629 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-67bb7fcf7f-cxv68_ee6796b4-e974-47e5-bf50-b5027d6a30c2/manager/0.log" Jul 06 20:12:29.800435 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:29.800403 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-756fc79d99-fvh2j_2369af8d-444e-488a-97c3-692ee5138a01/router/0.log" Jul 06 20:12:31.216132 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:31.216098 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-fdtsv_a1405983-5015-401d-a880-cb4abbd175f8/payload-pre-processing/0.log" Jul 06 20:12:31.270998 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:31.270945 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-756fc79d99-fvh2j_2369af8d-444e-488a-97c3-692ee5138a01/router/0.log" Jul 06 20:12:34.738783 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:34.738753 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-5cr4j_4a7e8265-6ce1-42c3-b589-43b0185cee0d/migrator/0.log" Jul 06 20:12:34.762360 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:34.762330 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-5cr4j_4a7e8265-6ce1-42c3-b589-43b0185cee0d/graceful-termination/0.log" Jul 06 20:12:35.116585 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:35.116492 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-6qvnx_2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112/kube-storage-version-migrator-operator/1.log" Jul 06 20:12:35.118286 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:35.118261 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-6qvnx_2512b8e7-36fe-4e92-9dc4-c1e0ae6d1112/kube-storage-version-migrator-operator/0.log" Jul 06 20:12:36.112497 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:36.112462 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9crmz_fcffc633-f4f2-4d7f-8fd2-d4c1819655dd/kube-multus/0.log" Jul 06 20:12:36.474588 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:36.474561 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zqg6b_e817f8b8-1d4a-445d-91c4-6c6d77d2bd28/kube-multus-additional-cni-plugins/0.log" Jul 06 20:12:36.498167 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:36.498137 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zqg6b_e817f8b8-1d4a-445d-91c4-6c6d77d2bd28/egress-router-binary-copy/0.log" Jul 06 20:12:36.523972 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:36.523944 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zqg6b_e817f8b8-1d4a-445d-91c4-6c6d77d2bd28/cni-plugins/0.log" Jul 06 20:12:36.559933 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:36.559869 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zqg6b_e817f8b8-1d4a-445d-91c4-6c6d77d2bd28/bond-cni-plugin/0.log" Jul 06 20:12:36.584012 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:36.583985 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zqg6b_e817f8b8-1d4a-445d-91c4-6c6d77d2bd28/routeoverride-cni/0.log" Jul 06 20:12:36.611671 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:36.611633 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zqg6b_e817f8b8-1d4a-445d-91c4-6c6d77d2bd28/whereabouts-cni-bincopy/0.log" Jul 06 20:12:36.641242 ip-10-0-131-35 kubenswrapper[2572]: I0706 20:12:36.641214 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zqg6b_e817f8b8-1d4a-445d-91c4-6c6d77d2bd28/whereabouts-cni/0.log"